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0 A recording medium, an apparatus for recording a moving image, an apparatus and a system for 
generating a digest of a moving image, and a metiiod of the same. 



® A system for generating a digest of a moving image according to ttie present invention includes: a 
generating section for generating a moving image having a plurality of frames, and for generating an operation 
signal in response to an Input received by tlie generating section during the generating of the moving Image; a 
priority assigning section for assigning a priority to each of the plurality of frames as a function of the operation 
signal; a selecfing section for selecting at least one image of a plurality of images corresponding to the plurality 
of frames, based on the priority; and an output section for outputting at least one image selected by the 
selecting section. 
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BACKOROUNP OF THE INVENTION 

1 . Field of ttie invention: 

The present invention relates to a recording medium, an apparatus for recording a moving image to tlie 
recording medium and an apparatus for generating a digest of ttie moving image in order to quickly and 
briefly see tlie contents of tlie moving image formed by a video camera or the like. Hereinafter, "a moving 
Image" is referred to as a group of a plurality of static images corresponding to successive frames, 
regardless of an object being moving or not 

2. Description of tlie Related Art 

Convention^ly, as a method for briefly grasping the contents of moving images recorded by a video 
camera or the ]\k&, there have been proposed a method in which images are displayed by using a rapid 
traverse operation of a video tape recorder (VTR) so that a prescribed number of frames are deleted at 
regular intervals, and a method in which images are displayed so that the reproduction ^eed can be 
manually controlled by using a variable speed reproduction operation of the VTR, However, according to the 
former method, the prescribed number of frames are deleted in-especfive of the contents of the respective 
frames, so that there arises a problem in that necessary frames may be omitted and unnecessary frames 
may be displayed despite the user's intent On the other hand, according to the later method, even though 
the images can be displayed according to the user's intent, a manual operation is required to be performed 
for a long time throughout the playback. 

Conventionally, as a method for generating digest images for a TV program, etc., there has been 
proposed a method in which the user previously sees all the recorded images and assigns a priortty to each 
frame of the recorded images, as is disclosed in the Japanese Laid-Open Publication No. 3-90968. The 
priority is determined by the user's judgrrtent. According to sucti a method, a digest of the images is 
displayed within a prescribed period of time. Herein, the priority means a degree of importance of each of 
the images based on the user's feelings. For example, the priority relates to the degree in which the user 
want to display the image as a digest compared with the other images and the degree in which the user 
want to display the image as a digest for a longer paiod of time. Even though such a method is useful for 
explaining the contents of the images to the other people, it is not useful to the user who want to know what 
lands of contents the images have recorded. This is because the user ceui grasp the contents of the images 
when the user asagns a priortty to each of the images. 

35 SUMMARY OF THE INVENTION 

A recording medium according to the resent invention includes: a moving image recording portion for 
recording a moving image having a plurality of cuts; and a management information recording portion for 
recording management inform.ation for managing the moving image recorded i.nto the moving image 

40 recording portion; wherein the management information recording portion includes a first portion for storing 
first information indicating a position at which one of the plurality of cuts is changed to another on© of the 
plurality of cuts in the moving image, and a second portion for storing second information indicating the 
priority of each of the plurality of cuts in the moving image. 

In one embodiment of the invention, the first information is an address on the recording medium, and 

45 ttie second infonmalion includes a time length of each of the plurality of cuts, and the degree of the change 
of the moving image within each of the plurality of cuts. 

In another aspect of the present invention, an apparatus for recording a moving image into a recording 
medium, includes; a management information gaierating section for receiving a moving image having a 
plurality of cuts, and for generating first information indicating the position at which one of the plurality of 

50 cuts is changed to another one of the plurality of cuts in the moving image, and second information 
indicaSng the priority of each of the plurality of cuts in the moving image, based on the moving image; a 
first recording section for receiving the moving image, and for recording the moving image into a recording 
medium; and a second recording section for receiving the first information and the second information 
generated by the management information generating section, and for recording the first information and 

65 the second information into the recording medium. 

In one embodiment of the invention, the management information generating section includes: a cut 
change detecting section for detecting an image existing at a position where one of the plurality of cuts is 
changed to another one of the plurality of cuts in the moving image; a first detecting secticai for detecting a 
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time length of a cut including the detected image based on ttie detected image; a second detecting secCon 
for detecting the degree of the change of the moving image within a cut including the detected image 
based on the detected image; and a third detecting section for detecting an address on the recording 
medium where the detected image is recorded. 

In another aspect of the present invention, an apparatus for generating a digest of a moving image, 
includes an input section for inputting a desired lime length of a digest of a moving image having a plurality 
of cuts; a receiving section, for receiving management information for managing the moving image; an 
extracting section for extracting at least one cut from the plurality of cuts in the moving image, based on the 
desired time length and the management information; a delennining section for determining the number of 
frames to be reproduced for each of the cuts extracted by the extracting section, based on the desired time 
length and the management information: and a selecting section for selecting an image of a frame 
determined by the determining section for each of the cuts extracted by the extracting section. 

In anotfier aspect of the present invention, a system for generating a digest of a moving image, 
Includes: a generating section for generating a moving image having a plurality of frames, and for 
generating an operation signal in response to an input received by the generating section during generating 
the moving image; a priority assigning section for assigning a priority to each of the plurality of frames as a 
function of the operation signal; a selecting section far sheeting at least one image of a plurality of Images 
corresponding to the plurality of frames, based on the priority: and an output section for outputting flie at 
least one image selected by the selecting section. 

In one embodiment of ttie invention, the generating section includes a camera, and the input is supplied 
to the generating section by tiie operator of the camera. 

In another embodiment of the invention, the system further includes: a section for recording the moving 
image having the plurality of frames and the priority corresponding to each of the plurality of frames into a 
recording medium; and a section for reading the moving image having the plurality of frames and the 
priority corresponding to each of the plurality of frames from the recording medium. 

In another embodiment of the invention, the system further indudes: a section for recording the rnoving 
image having the plurality of frames and the opwation signal into a recording medium; and a section for 
reading the moving image having the plurality of frames and the operation signal from the recording 
medium. 

In another embodimwil of the invention, the priority asagning section, includes: a section for storing a 
state of an image corresponding So a frsume previous to an input current frame; a section for receiving an 
input operation signal, and determining the stale of an image conresponding to the input current frame as a 
function of the input operation signal and the state of the image corre^onding to ttie previous frame; and a 
section for determining the priority of the input current frame as a function of the state of the image 
corresponding to the input current frame. 

In another embodiment of the invention, the priority assigning section includes: a plurality of first priority 
determining sections, each of the plurality of first priority determining sections including a section for Storing 
the state of an image corresponding to the frame previous to an input current frame, a section for receiwng 
an input operation signal and for determining the state of an image corresponding to the input current frame 
as a function of the operafion signal and the slate of the image corr©sp<WKiing to the previous frame, and a 
section for determining the first priority of the input cun^nt frame as a function of the state of the image 
corresponding to the input current frame; a w^ght calculating section coupled to each of the plurality of the 
first priority determining section, for calculating the weight for tfra first priority determined by each of the 
plurality of the first priority determining section; and a second priority determining section for defarmining 
. the second priority of the input current frame based on the respective first priorities which are weighted by 
the weight calculating section. 

In another aspect of the present inventicm, a method for generating a digest of a moving image, 
includes the steps of: generating a moving image having a plurality of frames; generating an operation 
signal in response to an ir^ut during generating tiie moving image; assigning a priority to each of tiie 
plur^ity of frames as a function of the operation signal; selecting at least one image of a plurality of images 
con-esponding to the plurality of frames, based on the priority; and outputting the selected at least one 
image. 

In one embodiment of the invention, the priority assigning step includes the steps of: storing a state of 
an image corresponding to a frame previous to an input current frame; receiving an input operation signal, 
and determining the state of an image corresponding tc tlie input current frame as a function of the input 
operation signal and the state of Uie image conresponding tc the previous frame; and determining a priority 
of the input current frame as a function of the state of tiie image corresponding to the input current frame. 
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In another embodiment of the invention, the priority assigning step includes the steps of: (a) determin- 
ing a first priority corresponding to a cun-ent frame as a function of each of a plurality Of input operation 
signals; (b) calculating the weight for each of the first priorities obtained by step (a); and (c) determining a 
second priority corresponding to the current frame based on the first priorities weighted by the step (b). 
5 In another aspect of the present Invention, an apparatus for recording a moving Image into a recording 
medium, includes: a generating secBon for generating a moving image having a plurality of frames, and for 
generating an op^ation signal In response to an input received by the generating section during generating 
the moving image; a priority assigning section for assigning a priority to each of the plurality of frames as a 
function of the operation signal; a recording section for the moving image having the plurality of frames and 
10 the priority assigned to each of the plurality of frames into a recording medium. 

in another aspect of the present invention, an apparatus for generating a digest of a moving image, 
includes: a priority assigning section for receiving an operation signal generated during generating a moving 
image having a plurality of frames, and assigning a priority to each of the plurality of frames as a function of 
the operation signal; a selecting section for receiving 1}ie moving image, and for selecting at least one 
IS image of a plurality of images corresponding to the plurality of frames, based on the priority; and an output 
section for outputting at least one image selected by the selecting section. 

In another aspect of the present invention, an apparatus for generating a digest of a moving image, 
includes: a motion vector detecting section for receiving a moving image having a plurality of frames, and 
for detecting a motion vector between a pwedetermined number of frames of the plurality of frames; a 
20 calculating section for calculafing at least one of the values of a zooming factor, the value of a rotation 
vector, a change degree of the zooming factor between the predetenmined number of frames, and a diange 
degree of the rotation vector between the predetermined number of frames, based on the motion vector; a 
shot breakpoint detecting section for detecting a shot breakpoint of the moving image, based on at least 
on© of the value of the zooming factor, the value of the rotation vector, the change degree of the zooming 
25 factor between the predetermined number of frames and the change degree of ttie rotation vector between 
the predetermined number of frames; a selecting section for selecting at least one image of a plurality of 
■"images corresponding to the plurality of frames, based on the shot breakpoint; and an output section for 
Outputting at least one image selected by the selecting section. 

In one embodiment of the invention, the moving image having a plurality of frames is recorded in a 
30 V. recording medium, and the motion vector detecting section includes a section for reading the moving Image 
frorfi the reccwrding medium. 

In. another embodiment of the Invention, the calculating section includes a section for recording at least 
one of the values of the zooming factor, the value erf the rotation vector, the change degree of the zooming 
factor between the predetermined number of frames, and the change degree of the rotation vector between 
35 the predetermined number of frames into a recording medium, and the shot breakpoint delecting seclfon 
includes a section for reading at least one of the values of the zooming factor, the value of the rotation 
vector, the change degree of the zooming factor between the predetermined number of frames, and tie 
change degree of the rotation vector betvyeen the predetermined number of frames from the recordirrg 
medium. 

40 In another aspect of the present invention, a recording medium, includes: a moving image recording 
portion for recording a moving image; and a management information recording portion for recording at 
least one of the camera operation InformatiiDn obtained by detecting a camera operation made by the 
operator during the recording of the moving image by the use of a camera, image processing information 
obtained by processing the moving image, and sensing information obtained by detecting a recording 

45 condition during the recording of the moving image by the .use of a sensor provided in the camera. 

In another aspect of the present invention, an apparatus for generating a digest of a moving image, 
includes: an information receiving section for receiving at ieast one of a camera operation infonmation and 
an image processing information, the camera operation information being obtained by detecting a camera 
operation made by an operator during the recording of a moving image by the use of a camera, the image 

so processing information being obtained by processing me moving image; a static image exfiracting section 
for detecting the time period from the recording start to the recording end based on at ieast one of the 
camera operation irifomnation and the image processing irrfonmalion, and extracting at least one static image 
of a plurality of static images Included In the moving Image recorded during the time period from the 
recording start to the recording end ba^d on at least one of the camera operations information and the 

66 image processing infomriation; ^d an output section for outputting at least one static image exSmcted by 
the extracting section. 

In one embodiment of the invention, the information receiving section further receives a sensing 
information obtained by detecting a recordng condition during the recording of the moving image by the 
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use of a sensor provided in the camera, ar,d the static image extractiiig section extracts at least one^atic 
image of the plurality of static images included in the moving Image recorded dur.ng the t.m8 Period frorn 
the recording start to the recording end based on at least one of the camera operation mformation, the 
imaae processing Information and the sensing information. „_,hi.^ 
In another aspect of the present invention, an apparatus for recording a movmg image onto a recording 
medium, includes: an infomiation acquiring section for acquiring at least one of 
information and an image processing information, the camera operation information being oWa,ned by 
dieting a o-era operation made by an operator during the recording of a moving image by the use of a 
camera, the image processing information being obtained by processing the movmg .mage a speeding 
section for detec^ng the time period from the recording start to the record ng end based on atleast one o^ 
the camera operation information and the image processing informat-on. for specifying at least one stajc 
mage of a plurality of static images Included in the moving Image recorded dunng the time penod f^m the 
r3ing start to the recording end based on at least one of ttie camera operation .nforrnafon and the 
m^ p^Sng information, and for generating infonnation on the specified at least one ^tf 
; L rSrding section for recording the moving image and the information on the specfed at least one static 

imaae into a recording medium. . , ^ - 

In one embodiment of the invention, the information acquiring section further acquires a sens^g 
information obt^dned by detecting a recording cor,dii,on during the recording of *Jf ^""^^ '7^^^''^^^ ^ 
use of a sensor provided in the camera, and the specifying section specifies at least one static image of the 
, plurality of static images included in the moving image recorded during the time period from the recording 
stelt to the recording end based on at least one of the camera operation information, the image processing 
information and the sensing information. .„„„„ 
In another aspect of the present invention, a method for generating a digest of a moving image, 
includes the steps of: receiving at least one of a camera operation information and an image Processing 
5 nformation. the camera operation information being obtained by detecting a camera operation made by an 
oSof during the recording of a moving image by the use of a camera, the image proce jing inforrnatio 
being obtained by processing the moving image; detecting the time penod ^^^'^ f^^'^^^l '^^^^ 
receding end based on at least one of the camera operabon information and Je 
informatron; extracting at least one static image of a plurality of ^t^t'^^^^es included in the « 
„ recorded during the time period from the recording start to the "^^^^^^^^^^^^^^^^ 
camera operation information and the Image processing information; and outputfang the extracted at least 

""'SlnfeTbodiment of the invention, the receiving step includes receiving a sensing information 
obtained by detecting a recording condition during the recording of the moving image oy the use o, a 
35 sensor provided in the camera, and the extracting step includes extracting at least one static image of the 
plurality of static Images included in the moving image recorded during the P^;"^ 
start to the recording end based on at least one of the camera operation information, the image processing 

information and the sensing information. 

Thu- the invention described herein makes possible the advantages of (1) providing a recording 
40 medium capable of be used when a digest of a moving image is generated and displayed, 2) providing an 
appSusTor recording a mo^Hng image into a recording medium capable of automatically generating a 
management information for managing a moving image and recording the managemen information m o the 
recording medium. (3) providing an apparatus and a system for generating and displaying a digest of^a 
rrvoving image capable of reflecting the user's intent during the recording of the moving image, so that the 
45 conterrts of the moving image can be quickly and briefly grasped. 

These and other advantages of the present invention will become apparent to those skilled in the art 
upon reading and understanding the following detailed description with reference to the accompanying 
figures. 

50 BRIEF D^CRIPTION OF THE DRAWIWOS 

Rgure 1 shows a recording medium according to a first example of the present invention. 

Figure 2 shows an example of a moving image including a plurality of frames. 

Fioure 3 shows a recording medium according to a second example of the present invention, 
ss Figure 4 is a block diagram showing a structure for an apparatus for recording a moving image 
according to a third example of the present invention. « „ r„f 

Rgum 5 is a block diagram showing structures for a cut-change detecting section and an inner cut 
image change degree detecting section according to the third example. 
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Figure 6 is a block diagram showing another structure for the cut-change detecting section according to 
the third example. 

Figure 7 is a block diagram showing a structure for an apparatus for generating a digest of a moving 
image according to a fourth example of tfie present Inv^rtion. 
5 Figure 8 is a block diagram showng a structure for a system for generating a digest of a moving Image 
according to a fifth example of the present invention. 

Figure 9A is a block cfiagram showing a structure for a part of the system according to the fifth 
example. 

Figure 9B is a block diagram showing a structure for another part of the system according to the fifth 
10 example. 

Figure 10 is a view for illustrating a relationship between image data, priority and frame number. 
Figure 11A is a block diagram showing a structure of a part of the system according to the fifth 
example. 

Figure 1 1B is a block diagram showng a stnjclure of another part of the system according to the fifth 
75 example. 

Figure 12A is a mode transition view with respect to a recording button group. 
Figure 12B is a mode transition view with respect to a zooming button group. 
Figure 13 is a block diagram showing a structure for a priority assigning section. 
Figure 14 is a view for illustrating how the priority of a mode is. changed by a number of successive 
20 frames. 

Figure 15 shows the relationship between the mode and the number of the successive frames. 
Figures ISA and 16B show a case whiere a priority of a START/STOP button and that of a zooming 
button are not overlapped with each other and a case where they are overlapped with each other. 
Figure 17 is a block diagram showing a structure for a priority assigning section; 
26 ''- Figure 18 Is a block diagram showing a structure for an apparatus for generating a digest of a moving 
image according to an sixth example of the present invention. 

' Figure 19A is a view showing the position of a lattice point for detecting a motion vector in the motion 
vector detecting section according to the sixth example. 

Figure 19B is a view shov«ng representative points with respect to Xne latSce point of Figure 19A. 
30 V :Figure 20 is a block diagram showing a structure for a motion vector detecting section accorcling to ihe 
slxf^i example. 

Rgure 21 shows the relationship between an imaging plane of a camwa and an object. 

■Figure 22A is a view for illustrating the change of the relationstilp between the image prfane of the 
camera and the object during a zooming operation. 
35 Rgure 22B is a view for illustrating the change of the relationship between the image plane of the 
camera and the object during a tilting operation. 

Figure 23A is a block diagram showing a structure for a shot breakpoint detecting section according to 
the sixth example. 

Figure 23B is a block diagram showing a structure for another shot breakpoint detecting section 
m according to the sixth example. 

Rgure 24A is a view for illustrating the 20C»ning factor f and a change amount thereof. 
Rgure 24B is a view for illustrating the x component px of a rotation vector P and the change amount 
thereof. 

Rgiwe 24C is a view for illustrating the y component py of the rotation vector P and the change amount 
45 thereof. 

Figure 24D is a wew for illustrating points of shot breakpoints. 

Figure 24E is a view for illustrating the period of time for an image to be selected as a digest image. 
Figure 25 is a block diagram showing a structure for a digest display control section according to the 
sixth example. 

50 Rgure 26 shows a recording medium according to a seventh example of the present invention. 

Figure 27 shows a structure for an apparatus for generating a digest of a moving image according to an 
eighfri example of the present invention. - 

Figure 28 shows a structure for a representative image information extracting section according to an 
ninth example of the present invention. 
55 Figure 29 is a block diagram showing another structure for a frame difference value detecting section 
shown in Figure 28. 

Figure 30 shows a structure for an apparatus for recording a moving image according to a tenth 
example of the present invention. 
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Figure 31 shows a Structure for a representative image extracting section according to a tenth example 
""^'^RguSlzA T^Se show waveforms of signals generated by the gate signal generating section 
according to the tenth example. , . .t. 

Figure 33 shows a structure for a representative Image extracting section according to a tenth example 
of the present invention. . j- _ . *„„tu 

Figure 34 shows a structure for another representative image extracting section according to a tenth 

example of the present invention. 

Figure 35 shows a example of a moving image including a pluraJity of shots. 

DESCRIP-nON OF THE PREFERRED EMBODIMENTS 

Hereinafter, the present invention will be described by way of illustrating examples. 

Figure 1 shows a recording medium according to a first example of the present invention. The 
recording medium is. for example, an optical disk 1. The optical disk 1 includes a plurality of tracks 2 for 
recording a moving image and at least one track 3 for recording management informafcon for managing the 
. moving Image recorded in the optical disk 1. Track 3 includes a portion for recording a positional 
information indicating the position at which a cut is switched to ano^er cirt m the 
priority information indicating the degree of priority of each cut included m the moving '^^^ ''^l^J'l'^^^ 
L positional infomiation include a -cut-change address" ol the tracks 2 of the optical disk 1. The pr o rty 
information includes a "cut priority" and an "inner cut priority". The terms of the "cut-change address", the 
5 "cut priority" and the -inner cut priority" are defined inthe next paragraph. 

As is shown in Figure 2, the moving image consists of a plurality of firames of sta ic images each of 
which is recorded as an analog signal or a digital signal in the tracks 2 of the optical *sk 1^ "A^^t is 
defined as a group of successive static images from a frame at which an input of a recording start is 
■ received by the imaging device to a frame at which an input of a recording end is '^^^'^J^^^?^^^^^ 
,0 device. "A cut-Change image" Is defined as a frame of static image at which a cut is switched to another 
cut For example, in Figure 2. it is assumed that a fii^t cut represeniins a mountain consists of frames (1 ) to 
(3) ■ a second cut representing a human body consists of frames (4) to (7). and a third cut representing a car 
consi^s of frames (8) and (9). In this case, the frames (4) and (8) are cut-change images. ^^^P^^^^^^J^^ 
contents of images are changed between the frames (3) and (4), and are changed between ihe irames (7) 
35 and (8). An address of the tracks 2 at which the cut-change image is recorded is hereinafter referred o as a 
"cut-Inge address". A length of a cut, i.e. the number of frames of a cut. is hereinafter referred to as a 
"cut priority". For example, in Figure 2, the cut priorities of the first, second and third cuts are 3. 4 and 2 
respectively The degree of the change within a cut is hereinafter referred to as an -inner cut priority For 
exaSe. I inner cut priority is represented by a mean value of the differences between frames inc^ded 
.0 in the same cut. Specifically, the inner cut priority is obtained by calculating the difference in a signal 
between the frames for each of corresponding pixels, by summing up the. difference over a corresponding 
pixels so as to be the summed up difference as a frame difference value, and by calculating the mean 
value of the frame differences value for all adjacent frames included in the same cut. Such an inner cut 
priority represents an average degree of the change of images between the frames in a cut. Accordingly, a 
45 Sut having a higher degree of the change of images means a cut in which the frames of images are 
^angrfn motion mom drastically. The cut-change address, the cut priority and the inner cut pnority are 
recorded in trac1< 3 of the optical disk 1. ^^^^ 
By recording the management information to the recording medium, it makes possible to ^i^y access 
a moving image recorded in the recording medium. For example, if only cuts having a prescribed value or 
60 more of the cut priority are selected from a moving image based on the management information and he 
selected cuts are displayed in a display apparatus, then one can quickly and briefly see the contents o he 
moving image which is recorded in the recording medium. In addition, one can easily find out the desired 
scene among the moving images based on the selected cuts. This is because a cut having a large vaiue of 
the cut priority can be considered to be representative of the contents of the entire moving image. Thus, 
.5 one can roughly see the contents of the entire moving image, for example, by extracting one frame from 
each cut having a large value of the cut priority and by displaying the extracted !rames in a display 
apparatus. Further, if only cuts having a prescribed value or more of the inner cut ("'^^'^ ^^J''^^^^^ 
a moving image based on the management information and the selected cuts are displayed in a display 
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apparatus, then one can know the portion where the. contents of images are drastically changed in motion 
among the moving image recoircied in the recording medium. 

As is described above, the recording medium of the first example of the present invention includes a 
portion ror recording a management information fcwr managing a moving image recorded in the recording 
6 medium, so that a user can readily find a desired scene. 

In the present example, the inner cui priority is represented by a mean vaiue of the differences between 
frames in the same cut, and the differences betA'een the frames are determined on the basis of the 
luminance values of the corresponding pixels between the frames. However, the inner cut priority may be 
obtained without using the differences between the frames nor the luminance values. For example, if a 
10 screen is divided into a plurality of blocks, arwi a motion vector for each of the plurality of blocks is 
calculated, the degree of variance of the motion vectors can be used as the Inner cut priority. 

In the present example, the record medium is an optical disk. However, the recording medium 
according to the present hvention is not limited to the optical disk. The present invention is also applicable 
to other types of recording mediums such as a solid memory like IC, a magnetic t^, or the like. 

Example 2 



Figure 3 shows a recording medium according to a second example of the present invention. The 
recording medium includes a first recording medium for recording a moving image and a second recording 

20 medium for recording a management information for managing the moving image recorded in the first 
recording medium. The first and second recording mediums are separated each other, whldi is different 
from the first example of the present invention. For example, as shown in Figure 3, the recording medium 
includes an optical disk 4 for recording, a moving image, an IC memory 5 fdr recording a management 
information for managing the moving image recorded in the optical disk 4, and a cartridge 6 for housing the 

25 optical disk 4 and the IC memory 5. The management informatton to be recorded in the IC memory is the 
same as that of the first example, and the description tt^ereof is omitted. 

■ ^ ^.According to the recording medium of the present example, the management information is recorded in 
tiie medium other than the medium for recording the moving image information, thereby making it possible 
to access the moving image quickly when the recorded moving image is reproduced. 

30 

Exannple 3 

' Figure 4 shows a configuration of an apparatus for recording a moving image according to a third 
example of the present invention. The apparatus includes an input section 7, an image signal processing 

35 section 8, a moving image recording section 9, a cut-change detecting section 10, a cut length detecting 
section 11, an inner cut-change degree detecting section 12, an address detecting section 13, and a 
management information recording section 14. 

An operation of the apparatus having the above configuration will be described below. A signal 
representing a moving image is input to the input section 7. The input signal is conveyed to the image 

40 signal processing section 8 and the cut-change detection section 10. The Image signal processing section 8 
modulates the input signal into a signal which Is suitable for a recording medium. The modulated signal is 
recorded in the recording medium by the moving image recording section 9. The cut-change detecting 
section 10 receives the input signal and detects a cut-change image at which a cut is changed into another 
cut in the moving image based on the input signal, A detecting signal representing a position of the cut- 

45 change image in the moving image is supplied lo the cut length detecting section 11, the inner cut-change 
degree detecting section 12 and the address detecting section 13, respectively. The cut length detecting 
section 11 detects the length of a cut in response to the detecting signal. The length of a cut is determined 
by counting the number of franes included in the cut. The inner cut-change degree detecting section 12 
detects the degree of the change between the frames included in the cut in response to the detecting 

50 signal. The address detecting secfion 13 detects the address of the recording medium at which the cut- 
. change image detected by the cut-diange detecting section 10 is recorded in response to the detecting 
signal. The management information recording section 14 records the length of *e cut, the degree of the 
change and the address of the cut-change image'in the recording medium as a management information for 
managing the moving image recorded in the recording medium. 

65 Next, a further detail configuration of the cut-change detecting section 10 and the inner cut-change 
degree detecting section 12 will be described with reference to Figure 5. The cut-change defecting section 
10 includes a frame memory 101 for delaying an input signal representing a moving image by one frame, a 
difference detecting section 102 for calculating tiie difference between successive frames, and a cut-change 

8 
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iudQina section 103 for iudging whether a cut-change occurs or not by comparing the difference between 
TZc^TeiZesylm^ a'pradeterminad value. The cut-change iudging section 103 judges that a cut- 
change occurs if the difference between the frames is equal to. or greater than the predetermined value 
The inner cut-change degree detecting section 12 includes the frarr^e memory 101 for delaying the input 
signalby one frame, the difference detecting section 1Q2 for calculating the difference between successive 
frames an addition section 121 for summing up the differences between the frames in the same cut, and an 
average section 122 for calculating the mean value of the summed differences between the_ frames m fte 
same cut by dividing the sum of differences between the frames obtained by the adding section 121 by the 
number of frames detected by the cut length detecting section 11. For example, a lum.nance value and rgb 
value can be used as a signal for calculating the difference between the successive trames. By providing 
the configurations shown in Figure 5, the cut-ohange detecting section 10 iudg^^that a cut-change occurs 
at the portion where the difference of the adjacent frames is large, since it is likely to greatly change the 
contents of the adjacent frames at that portion. The inner cut-change degree detecting section ia calculates 
the mean value of the summed differences between the successive frames within the cut as the degree in 
the change between the successive frames. , c- c *h„ 

Figure B shows another configuration of the cut-change detecting section 10. In Figure 6. the cut- 
change detecting section 10 includes a color histogram detecting section 104 for detecbng a ooor 
histogram of an input signal representing a moving image for every frame, a "^'^r^^-^^^irH . thl 
recordinq the detected color histogram, a histogram difference detecbng section 106 for detecbng the 
difference of color histograms between tt»e respective successive frames, and a cut-change judging section 
107 for iudging whetheTa cut-change occurs or. not by comparing the difference of the color h^tograms 
beLeen the successive frames within a predetermined value. The cut-change judging section 107 judges 
that a cut-change occurs if the difference of the color histograms between the frames is equal to. or greater 
than the oredetermlned value. In the cut-change delecting section 10 shown in Figure 6. a cotor histogram 
in a frame is used as a signal for detecting a cutKJhange. so that the cut-change detecting peribnnance will 
not be affected by the change between the frames. ^ ..^ ^ „„™»™<>nt 

Thus according to the apparatus for recording a moving image mentioned above, the management 
information for managing the moving image is automatically generated from the i"P^^'"0^'"9 ""^j^ 
recoS into a recording medium at the same time when the moving image is recorded mto the recording 
medium. Specifically, the apparatus automatically detects a cut-change image among the moving image 
JetS^ihe address ^f the recording medium at which the cut-change image is recorxled the tme length of 
each cut. and the degree of the change between the frames in each cut. and records the address of he 
cut-change, the time length of each cut and the degree of the change between fi-ames .n ea<^ cut mto the 
recording medium as managing inftomation. This makes it possible to add the management information to a 
l^ge amoum of moving image data without any humw help or assistance. 

Example 4 

Fioure 7 shows a configuration of an apparatus for generating a digest of a moving image according to 
a fourth example of the present Invention. The apparatus includes a time length input section 16 a 
management information reading section 17, a cut extracting section 18, a reproduced frame number 
deciaon section 19, and a reproduced image retrieval section 20. 

Next, the operation of the apparatus having the above configuration will be described, A management 
information for managing a moving image, as well as the moving image, is recorded in a recording medium 
, 15, as described in the first and second examples. 

The apparatus generates a digest of a moving image having arbitrary time length specified by a user 
based on the management information. The management information reading section 17 r^s a manag^ 
ment information from a recording medium. Alternatively, the management infomrtalion reading section 17 
may receive the management information from any device such as an imaging device instead of reading 
, the management information from the recording medium. The management information includes an address 
at which cut-change images is recorded, the length of a cut and the degree of the change in a cut. 

The desired time length of a digest of the moving image is input to the time length input section 16. 
The Input time length is conveyed to the cut extracting section 18 and the reproduced frame number 
decision section 19. The cut extracting sectio. 13 e>rtracts at least one cut from the plurality of cuts 
e included . in the moving image based on the managen^e.! information and the input time length. The 
reproduced frame number decision section 19 determines the number of frames to be reproduced for each 
of the extracted cuts based on the management infomnation and the input time length. Finally, the 
reproduced image retrieval section 20 retrieves frames for each of the cuts extracted by the cut extracting 
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section 18 from the recording medium 15 in accordance with the number of frames to be reproduced for 
each of the extracted cuts determined by flie reproduced frame number decision section 19. 

Hereinafter, the operations of the cut extracting section 18 and the reproduced frame number decision 
section 19 wtfl be described In detail. Herein, n denotes the number of cuts included in the moving Image 
5 which Is recorded in the recording medium, T (sec.) denotes the time length input to the time length Input 
section 16 by the user, and m denotes the minimum number of frames of respective cuts required to be 
seen as a animated film. 

As a first case. It Is assumed that the input time length T Is short. Specifically, it Is assumed that the 
Input time length T satisfies the following relationship shown in the equation <1). 

10 

TSn.m.t (1) ■ 

wherein, t denotes the time interval between respective two frames. 

In this case, If the cut extracting sectiort 13 extracts m frames from each of cuts included In the moving 
?s image, 1he time length of a digest consisting of the extracted frames would exceed the input time iength T. 
In order to restrict the time length of tfie digest within the input time length T, the cut extracting section 18 
determines the number of cuts n' to be extracted in accordance with the followring equation (2). 

n' = T/(m.t) (2) 

If the number n' is not an Integral value, the number n' is rounded up or down so as to be an Integra 
value. 

The cut e)ctracting section 18 determines the number n' in accordance with the equation (2), and then 
extracts n' cuts vyhich have a longer time length from the n cuts included in the moving image in the order 
25 from a cut having the longest time length to a cut having the shortest time iength, based on the 
management information which is read from the recording medium 15. The reproduced frame number 
•decision section 19 determines to malte the number of frames to be reproduced for each of the extracted n' 
cut to be m. Then, the reproduced image retrieval section 20 retrieves m frames of the respective n' cuts 
from the recording medium 15 based on addresses of n' cut-change images extracted by the cut extracting 
30 section 18. 

As a second case, it is assumed that the input time length T is long. Specifically, it is assumed that the 
input time length T satisfies the follovnng relationship shown in the equation (3). 

r>n.m.t (3) 

35 

In this case, even if the cut extracting section 18 extracts m frames from each of the n cuts included in 
the moving image, frie time length of a digest consisting of the extracted frames is Sorter Uian the input 
time length T. Accordingly, the number of frames m" to be further extracted from the n cuts is defarmlned 
in accordance with the folbwing equation (4). 

40 

m" = T/t- n-m (4) 

The reproduced frame number decision section 19 detemrtines the number m" in accordance witti the 
equation (4), and then determines the number of frames m;' to be reproduced for each of the n cuts, in 
45 accordance with the following equaSon (5). 



m^' - ni + m'-x^ /2^x^ ... (5) 

60 *• ' . . 

wherein, I denotes an identifier to identify eacJi cut, and xi denotes the degree of the image change 
within the ith cut. If the number ms' is not an integral value, the number mi' is rounded up or down so as to 
be an Integral value. Then, the reproduced image retrieval section 20 retrieves frames of the respective 
S5 n cuts from the recording medium 15 based on the addresses of ttie n cut-change images extracted by the 
cut extracting section 18. 

According to the apparatus of this example of the present invention, it is possible to automatically 
generate a digest having a time leng^ desired by tine user for a rapid traverse operation, based on the 
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input desired time length of the digest and the management information. 



Example 5 

RgufB 8 shows a configuration of a system for generaiing a digest of a moving image according to a 
fifth example of the present invention. The system calculates the priority of each frame and recpnds the 
priority in a recording medium during making a moving image by the use of a camera. Then, the system 
selects at least one frame of the frames included in the moving image based on the priority recorded m the 
recording medium and displays images corresponding to the selected frames as a digest of the moving 
Image For this reason, first the operation of the system in making a moving image will be described v«th 
reference to Figure 9A and the second operation of the system in displaying a digest of the moving image 
will be described with referring to Figure 9B. 

Figure 9A shows a part of the configuration of the system for generating a digest of a moving .mage. 
This part calculates the priority of each frame and records the priority in a recording medium dunng making 
a moving image by the use of a camera. . , ^- ,^ ■ 

in Figure 9A the system includes a camera 201 for outputting a.video signal, a vertical/horizontal scan 
synchronous signal, and a button input signal conresponding to a button operation of the camera 201, and a 
priority assigning section 205 for assigning a priority to each of frames in the moving image based on the 
button input signal from the camera 201. ' . ^ , „ w^^i^^ 

The video signal and a verticalfliorizontal scan synchronous signal may be received from an imaging 
device. The button input signal is. for example, a signal corresponding to a recording start ^operation or a 
zooming operation. Hereinafter, it is assumed that the button input signal has a value of 1 when the button 
is pressed so as to be in the ON-stale, while the button input signal has a value of 0 when the button is not 
pressed so as to be in the OFF-state. A method for assigning a priority to each frames will be described 

* A database 202 Includes two recording mediums. One is a moving image recording medium 203 for 
recording the video signal and the synchronous signal output from the.camera 201..The other iS a pnonty 
recording medium 204 for recording the priority output from the priority assignir»g secbon 20S The 
database 202 has the function of recording data into the recording mediums as well as the function of 
reading the recorded data from the recording mediums. 

Raure 10 schematically shows the relationship between the image data and the priority corresponding 
to the image data. In Figure ID. the frame number (F) represents an identifier for identifying each frame 
included in the moving image. For example, the frame number (F) is a serial number of the frariies. The 
image data corresponding to each frame is recorded on the moving image recording medium 203. I he 
priority corresponding to each frame is recorded on the priority recoroing medium 204. ^ 

Figure 9B shows a part of the configuration of the system for generating a digest of a moving image. 
This part displays a digest of the moving image based on the priority corresponding to each frame. The 
database 202, the moving Image recording medlurn 203, and the priority recording medium 204 are the 

same as those of Figure 9A. ^. ^ i 

When a START/STOP input section 210 receives an instnjction tor a start of displaying a digest of the 
moving image, the frame number of the priority to be processed in the priority recording medium 204 is 
made to meet the frame number of the video signal to be processed initially in the moving image recording 
medium 203. The image selecting section 211 specifies at least one frame of the frames in the moving 
image based on the priority, and selects at least one image corresponding to the specrfied frame of al 
images included in the moving Image. The output section 215 displays me selected images as a digest of 
the moving Image. The output section may be a display or a printer. 

Hereinafter, the processing at the image selecting section 211 will be descnbed in detail. 
The priority corresponding to each frame is input to the image selecting section 211. A judg- 
ment'synchronous section 214 compares the input priority with a threshold value. The threshold value is 
stored in a threshold value memory section 213. When a priority corresponding to a current frame is higher 
than the threshold value, the judgment/synchronous section 214 selects an image corresponding to the 
current frame of all images In the moving image. The selected image is conveyed to the output section 215 
If the threshold value Is set to a higher value, the number of the selected images is decreased, whereas i 
the threshold vaiue Is set to a lower value, the number of the selected Images is Increased. Thus, the level 
of the threshold value can be used as a measure for reducing the amount of Information. Only the images 
corresponding to the priority higher than the threshold value are displayed as a digest of themoving image^ 
Then, the judgment/synchronous section 214 transmits a response signal to the database 202 so as to ask 
the database 202 to output a video signal and a priority corresponding to the next frame. The operation 
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mentioned above is one cycle of the operation with respect to the current frame of image. By repeating 
such a cycle for each of the frames until the START/STOP input section 210 receives an instruction to stop 
disp!ayir>g the digest, ttie selected images are displayed as a digest of the moving image. Further, the 
output section 215 may be a recording device for recording a digest of the moving image instead of 
5 displaying the digest or printing the digest. 

in order fo change the threshold value, a threshold value changing section 212 may be provided in the 
image selecting section 211. The threshold value changing section 212 receives a hew threshold value and 
changes the threshold value stored in the threshold value memory section 213 into the new threshold value. 

Thus, according to the system of this example of the present Invention, it is possible to automatically 
10 display a digest of the moving image by determining the priority of the respective images based on the 
button input sign^ during the recording operation. 

In the present example, only the images that eadi have a higher priority are selected and displayed in 
the sequence order. Alternatively, when a screen is divided into a plurality of blocks, the images each 
having a higher priority may be displayed on respective blocks simultaneously, and when those blocks are 
76 different to each other in size, the images each having a higher priority may be displayed so that the 
images each having a higher priority are allocated to larger btocks, and the like. 

In the present example, the priority is calculated during recording a video signal. However, the same 
effects can be obtained when the button input signal is recorded during recording a video signal, and then 
Uie priority is calculated on the basis of the recorded button input signal during the reproduction of a digest 
30 of the moving image. Such an example v/ill be described belov^ with referring to Figures 11A and 11B. 

Figure 11A shows a part of another configuration of ttie system for generating a digest of a moving 
image. This part records a button input signal on a recording medium during the recording of the moving 
image. A camera 201 and a moving image recording medium 203 are as the same components as those 
shown in Figure 9A. 

25 The system includes a camera 201 for outputting a video signal, a synchronous signal, and a button 
input signal- The video signal and the synchronous signal may be received from an imaging element. A 
database 202 includes two recording mediums, a moving image recording medium 203 for recording the 
video signal and the synchronous Signal from the camera 201, and a button input signal recording medium 
206 for recording the button input signal and the synchronous signal. 

30 Figure 11B shows a part of another configuration of the system for generating a digest of a moving 
image. This part determines the priority for each of the frames based on a button input signal recorded In 
the button input signal recording medium 206, End then displays a digest of the moving image based on 
the priority, A START/STOP input section 210, a moving image recording medium 203, an image selecting 
section 211. and an output section 215 are the same comiDonents as those of Figure 9B. A priority 

35 assigning section 205 is the same component as that of Figure 9A. 

The operation of ttie system in displaying a digest of the moving image will be described below. When 
the START/STOP input section 210 receives a signal for the start of displaying a digest, the frame number 
of the button input signal to be processed in the button input signal recording medium 206 is made to meet 
the frame number of the video signal to be processed initially in the moving image recording medium 203. 

40 The priority assigning section 205 asagns a priority to each of frames based on the button input signal 
which is recorded on the button input signal recording medium 206. Then the image selecting section 211 
specifies at least one frame of all of the frames in the moving image based on the priority, and selects 
images corresponding to the specified frames as a digest of the moving image, in the same manner as 
mentioned above. The selected images as a digest of the moving image are displayed by the output 

45 section 215 such as a display. 

Thus, according to the system of this example of the present invention, a digest of the moving image 
can be displayed by recording the button input signal during recording a video signal and by determining 
the priority during reproducing the digest. 

Hereinafter, in order to clarify how to assign a priority to eadi of the frames in the priority assigning 

50 section 205 shown in Figures 8, 9A and 11B, the principle of assigning a prioilty to each of the frames 
based on a button input signal will be described below. 

The video camera is provided with various buttons such as a recording button, zooming button, fade 
button, focus button, exposure btJtton, white balance button, shutter speed button, etc. It is possible to 
assume the user's intent in recording a moving image by detecting which button is pressed. The user's 

as intent is, for example, directed to an image which the user considers very important in recording the image 
and a cut-change image at which a cut is changed into another cut. If it is detected that the START/STOP 
button or the fade button has been pressed, it is clear that a cut is changed. If It is detected that the 
zooming button is in the ON-state, ft is assumed that the user is adjusting an angle of view. If an object is 



13 



EP 0 597 450 A2 



imaged in a close-up after the angle of view Is set. there is a strong likelihood that the user considers the 
image of the object very important. 

In order to make a digest for quickly and briefly seeing the contents of a moving image, there is a 
method in which only important Images are displayed and another method in which more important images 
are dismayed for a longer time. In order to make a digest including all cuts of the moving image, for 
example there is a method in which images are extracted from each of the cuts for a prescribed period of 
time and the extracted images are displayed. Any one of these methods can be realized by adjusting the 
priority assigned to each of the frames in the moving image. In the case where only images having a 
priority of not less than a prescribed value are displayed as is done in the image selecting section 211 of 
Figure 9B, if a more important image is assigned to a higher priority, it becomes easy to display the 
important image, whereas it becomes very difficult to display a non-important image. 

As is described above, if the state of Image such as an important Image and a cut-change image is 
assumed based on the button operations by the user, and a method of displaying a digest is determined, a 
digest by which the contents of the moving image is quickly and briefly seen will bo obtained. 

Table 1 shows the relationship between button operations and the state of images, and the relationship 
between the state of Images and the priorities. \With reference to Table 1, a method for assigning priority to 
each of frames based on the button operations will be described. In Table 1. the buttons are classified into 
two groups: a recording button- group Including the recording button by which the cut-change can be 
detected and a zooming button group including the zooming button by which the important images can be 
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The two groups each have a different number of button input sigr^ais. Specifically, a button in the 
recording button group generates two patterns of button input signals. This is because the button in the 
recording button group is a singie input button and is in either the ON-state or the OFF-state. Herein, when 
the input button is in the ON-state. the state of an image is in a state of the adjustment of the image. Just 
after the input button is changed from the ON-state to the OFF-state, the state of the image is In a state of 
the cut-change, A button in the zooming button group generates more patterns of button input signals. This 
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is because the button in the zooming button group has two input buttons and each input button is in either 
the ON-state or the OFF-state with the combinations. One input buttai of the zooming button group is 
called a set button such as a zooming-in button and a serial shooting setting button. The other input button 
of the zooming button group is called a release button such as a zooming-out button and a sena! shooting 
releasing button. However, generally, both of the set buttons and the release buttons would not be in the 
ON-state, therefore, there are three kinds of input patterns as follows: 

ON-state of the set button. 

ON-state of the release button, and 

hlreirJhen the set button is in the ON-stats. the state of the image is in a state of the adjustment of 
the image. When the release button is in the ON-state. the state of the image is also in a state of me 
adjustment of the image. During a period from the end of the ON-state of the set button to me start of the 
ON-state of the release button, the state of the image is in a state of an important image. 

As mentioned above, the state of the image is assumed from me button input signal. The state of me 
image includes a state of me cut-change, state of me adjustment of me image and a state of me important 

'"^^Twitl be described below how a priority is determined in accordance with the state of the image. 

The priority of an image which is recorded during the adjustment of the angle of view is made lower so 
as to be omitted from displaying a digest. On me other hand, the priority of an important irnage is made 
higher so as to be selected as a digest of me moving image. In order to make a digest including a I cuts a 
prescribed number (TH) of images after me cut^change are supplied wim higher priority so that at least the 
first images of me respective cuts are displayed. Tfie priority of images omer than the cut-change image, 
the image during the adjustment of the angle of view, and me important image are made at 0. 

Hereinafter a period of the image during which a priority is assigned to frames by me use of me same 
rule is referred to as "a mode". It will be described how me priority is assigned to frames, as follows: 

The recording button group (three modes of A. B and C below): 

A; ON mode (me button Is in the ON-state: me state of the adjustment an image; prionty P < 0) 
B: cut mode (a period Of time of TH frames starting lirom a cut-change Image; priority P > 0) 
C; normal mode (omers; priority P = 0) 
The zooming button group (four modes of A, B. C and D betow); . ^ ^ 

A: set-ON mode (me set button is in me ON-state: the state of me adjustment of an image; priority P < 

B: release-ON mode (the release button is in the ON-state: me :StaiB of me adjustment of an image; 

priority P < 0) ^ d 

C: important mode (me state of importarvt image such as clo«e-up shots; prionty P > O) 

D: normal mode (others; priority P = 0) 
Hereinafter, me memod for assigning a priority to frames will be described more concretely wrth 
reference fo Table 1, Figures 12A and 12B. Rgure 12A shows a mode transition of images wim respect to 
the recording button group. Figure 12B shows a mode transition of images wim respect to the zooming 

"^""Asl^s^n from Figures 12A and 12B. me mode of me currently processed image can be determined 
frc«n the foltowings items: 

(1) a mode of the previous frame. 

(2) an ON or OFF state of me Ixrtton, and 

(3) the number of frames by which me same cut mode lasts (only in a case of the recording button 

^AsTs described above, the current mode of the image is determined in accordance wim the procedure 
of the mode transition of the image shown in Figures 12A and 12B. and me priority of me image is set so 
as to correspond to me mode. Thus, me priority of respective images can be determined from me button 

'"'^"Figure'is shows a configuration of the priority assigning section 205 for assigning a priority to each of 
the frames In the moving image based on me button in put signal in the recording button group according 
to the above-mentioned principle. The priority assigning section 205 includes a state detecting section 220 
■ for determining the mode of a current frame, and a priority determination section 224 for generating a 
: priority as a function of the mode determined by me state detecting section 220. The state detecting 
section 220 includes a mode accumulating section 221, a mode judging section 222, and a counter 223. 
First, the operation of the state detecting section 220 will be described. 
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The mode accumulating section 221 stores a mode of an image corresponding to tne previous frame. A 
counter 223 stores the number <Fc) of frames by whicli the same mode lasts. The mode of the image 
corresponding to the previous frame and the number Fc are supplied to the mode judging section 222. 
Initially, the mode accumulating section 221 is set to a normal mode, and the counter 223 is set to 0. 
5 The mode judging section 222 receives an button input signal from the camera 201, and determines the 
mode of the image corresponding to the cun-ent frame based on the button input signal, the state of the 
image corresponding to the previous frame and the number Fc in accordance with the state transition 
shown in Figure 12A. After determining the mode of the image corresponding to the current frame, the 
mode stored in the mode accumulating section 221 is updated. Furthermore, the mode judging section 222 
10 resets tiie number Fc to 0 when the mode is changed, and increments the number Fc one by one for every 
frame as long as the same mode lasts. 

The priority determination section. 224 delemnines the priority corresponding the current frame as a 
function of the mode of the image corresponding to the current frame determined by the mode judging 
section 222. Speclflcaily, the priority P is detemiined using a function K as follows: 

P = K(M) 

wherein, M denotes the current mode, and K denotes, for example, a ftinction with which P is converted 
to be -1 if the ON mode, 0 if the normal mode, and 1 if the cut mode. 
so In order to assign a priority to each of the frames based on an button input signal in the zooming button 
group, the configuration of the priority assigning section 205 shown in Rgure 13 may be modified to the 
foilowng manners: 

(1) the procedure shown in Figure 12A in the mode judging section 222 may be modified into a 
procedure shown in Figure 12B, and 
25 i2) the counter 223 shown in Rgure 13 may be omitted. This is because the nimnber of the franes by 
which the same cut mode lasts is not used in judging the current mode. 

As is described above, according to tne present example, by assuming the state of the image on the 
basis of the button Input signal, and a priority which meet the user's intent can be set to each of the frames, 
so that a digest can be generated to meet the user's intent. . 
30 In the present example,,ttie priority to respective modes is constantly maintained, but it is also possible 
to vary tiie value of the priority as time elapses as Is shown in Figure 14. Moreover, it is also possible to 
continuously adjust the display time of ttie digest by the threshold value changing section 212 of Rgure 9B. 
iri this case, the priority determination section 224 determines the priority using the following equation: 

35 P = K{M).L(Fc, M) 

The relationships between L(Fc, M) and Fc are shown in Figure 15. 

Hereinafter, another configuration o1 the priority assigning section 205 will be described. The priority 
assigning section 205 having the configuration shown in Figure 13 assigns a priority to each of the frames 

40 on the basis of a single button input sign^. Now, the priority asagning section 205 for assigning a priority 
to each of the frames on the basis of any combination of a plurality of button input signals. 

It is assumed that a priority P is independently determined on the basts of respective button input 
signals. Figure 16 shows time charts where the horizontal axis shows a number of frames, and the vertical 
axis shows respective button Input signals of the recording button and the zooming button, and respective 

45 priorities thereof which are determined according to the previous example on the basis of the respective 
button input signals thereof. Rgure 16A shows a case where the priority of the recording button and that of 
the zooming button are not overlapped with each other, in this case, only the pnority of one button Is not 0, 
and the priorities of the others are 0. Therefore, «ie priority not equal to 0 can be regarded as a final value. 
However, generally, as is shown in Figure 16B. the pnority of the recording button and that of the zooming 

50 buttCHi are overlapped with each other at one or more portions. 

According to the present example, a mode (hereinafter, referred to as a button operation mode in 
distinction from the mode referred in the previous example) is independently judged on the basis of the 
respective buttons, and finally ail the priorities of respective modes are integrated. 

Figure 17 shows another configuration of the priority assigning section 205. The priority assigning 

S5 section 205 includes a plurality of mode judging unit 231 and a weight calculating section 232 and a final 
priority determination section 233. Each one of the plurality of mode judging units 231 is the same as the 
unit shown in Rgure 13 in structure, and con^sponds to each of the plurality of button input signals. 
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The button input signal from the camera 201 is input to ttie mode judging unit 231 provided per button. 
When a particular button is in the ON-state, the button input signal corresponding to the particular button ,s 
supplied to each of the plurality of mode judging unit 231 so that the button signal has a value of fo 
Z one mode judging unit 231 corresponding to the particular button and the button s.gnal has a value of 
"S'^for L 0 J mode judging units 231. The operation of the mode judging unit 231 is --^^^^^^^ - 
that of Figure 13. Specifically, the button operation state delecting section 234 detects a button operaton 
mode M according to the mode transition of Figures 12A and 12B. The prionty delermmation section 235 
converts the button operation mode M to one of the values {-1. 0. 1} using the funcfaon K. 

Thus, the output of mode judging unit 231 Is obtained per button. Then, .t ,s detemnined ..h.ch pr.orrty 
of button is employed as the final priority on the basis of various factors as is described betow. 

As a first exemplary factor, if the cut mode of the recording button and the set ON rr>octe of the 
zooming button are overlapped as Is shown In Figure 16B. the zooming button which was .nput later is 
considered to reflect tfie user's intent better. 

AS a second exemplary factor, the rooming button and the serial shooting button are >^olh considered to 
be used for important images. However, the image corresponding to the senal shooting button ,s 
considered to be more, important than the image corresponding to the zoom.ng button. This is because the 
serial button is generally less used. Furthermore, ihe recording button and the exposure button are both 
considered to be used for changing the cut. but there Is a case where the cut remains unchanged even the 

1,-OIIBIUorDU i>j u a ^ ... , .... «,„ ,^^,Amr, hiiHnn a nmssed Thus. 



^ru^buttoTirpressed! whlirthe'cut' is^alw^^ changed when the recording button is pressed. Thus^ 
even if the same state of the image can be assumed from different button operations, the probability that 
such an assumption meets the user's intent varies depending on the kind of button pressed. . 

in the present example, in order to determine the final priority from a plurality of prior.tes determined 
by the m^e judging unit 231 per button, a weight per taitton is introduced. The we.ght per button is 
represented by ttie following equations with regard to the above factors: 

iv, (i: the serial number of a button, 1 S i S B, B = total number of buttons. the serial number of a 
factor, 1 S j S C, C = total number of factors) . .„ ^ u ^ 

If the above first exemplary factor is defined as "a mode updated more recently, will have a heavier 
weight", the weight for a button I can exemplarily be represented as follows: 



wherein. Fc{l) denotes the number of frames by which the button i is in the same mode, and a denotes 

^"^If^fabove second exemplary factor is defined as "a button having a stronger likelihood tiiat the 
assumed result meets the user's intent will have a heavier weight", weights for the recording button (1) the 
exposure button (2), the serial shooting button (3), and the zoommQ button (4) ^\^^^^°f''^^J^^^l 
example, at w12 = 1. w22 = 0,6, w32 = 1, and w42 = 0.8, respectively. Tlie number in ttie parentheses 
represents a serial number for the button. 
- m Figure 17. the button operation mode for the button , is represented as M, and t^e outpirt of the 
mode judging unit 231 is represented as K{M,). The weight calculating secton 232 calculates the weight W, 
for the button ion ttie basis of the weight wij of the button iwitti respect to the factor J as follows: 

Wi " Tf «i j 



The weighted priority Pi of the button 1 is given by ttte following equation: 

50 P, = K(Mi)-Wi 

The priority detemiination section 233 compares absolute values |Pi| of respective weighted priorities 
per butlo^i. and then selects a button i... having a maximum value [Pi...!. Thus, the pnonty P^.^ of .he 
button i„ax Is en^toyed as a. final priority. , ■ 

.6 As ^ described above, the final priority is determined from ttie plurality of weighted priorities per 
button and consequently. Itie user's intent can be assumed better from the various conditions. Accordlngh^ 
on the basis of the thus determined final priority, the digest images can be selected, and images r^arded 
as important by the user can be displayed, so that ttie contents of the images can be roughly grasped. 
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Example 6 

An apparatus for generating a digest of a moving image according to an sixtti example will be 
described. The apparatus includes a portion for generating a digest and a portion for displaying a digest. In 

s the apparatus, "a process of generating a digest" is executed prior to executing a process of displaying the 
digest. In the process of generating a digest, a camera operation during the recording of a moving image is 
assumed and is recorded In a database. Then, a digest is displayed based on the recorded camera 
operation in a process of displaying a digest. The camera operation includes any operation for changing 
direction, a position and an angle of view of the camera, and any combination thereof, such as panning, 

10 zooming, or the like. 

Figure 18 schematically shows a configuration of an apparatus for generating a digest of a moving 
image according to the present example. The apparatus includes a digest generating portion and a digest 
displaying portion. The digest generating portion includes a motion vector defecting section 304 for 
detecting a motion vector between the frames in a moving image artd a camera operation assuming section 

JS 305 for assuming a camera operation based on the motion vector. Tfie digest displaying portion includes a 
shot breakpoint detecting section 307, a digest display control section 308 and a output section 309. In 
Figure IS, a database 302 including a moving image recording medium 303 and a camera operation 
recording medium 306 is provided between the digest generating portion and the digest displaying portion. 
However, it is not necessary to provide such a database, if infonmatlon is exchanged directly between the 

so digest generating portion and the digest displaying portions. To swilEh a mode of generating/displaying a 
digest, the apparatus may include a digest generating/displaying selecting section 301 for recemng an Input 
from a user. 

First, the process of generating a digest in the digest generating portion will be described. 
When a mode of "generating a digest" is set in the digest generating/displaying selecting section 301, 
S5 the moving Image recording medium 303 is put into a state where the motion vector detecting section 304 
can read data from the moving image recording medium 303, and the camera operation recording medium 
306 'Is put into a slate where the camera operation assuming section 305 can write data into the camera 
operation recording mediurrs 306. 

The motion vector detecting section 304 receives a video signal from the moving image recording 
30 medium 303, and then detects a motion vector between the frames for conresponding local portions of a 
screen. The detected motibn vector is input 1o the camera operation assuming section 305. The camera 
operation assuming section 305 detects the operational amount of panning, zooming, or the like during the 
recording of a moving image so as to assume a camera operation. The detected operation amount is 
recorded into the camera operation recording medium 306. 
35 Second, the process of displaying a digest in the digest display portion will be described. 

When a mode of "displaying a digest" is set in the digest generating/displaying selecting section 301, 
the moving image recording medium 303 is put into a stale where the digest display control section 308 
can read data from the moving image recording medium 303, and the camera operation recording medium 
306 is put into a state where the shot break-point detecting section 307 can read data from the camera 
40 operation recording medium 306. 

The shot breakpoint detecting section 307 detects a frame corresponding to a breakpoint of ttie camera 
operation based on the operation amount recorded in the camera operation recording medium 306. For 
example, a start of the zooming operation and an end of the panning operation represents a breakpoint of 
the camera operation. Hereinafter, "a shot breakpoint" is referred to as a frame corresponding to a 
45 breakpoint of the camera operation. 

Figure K shows an example of a moving image. The moving image includes a plurality of images 
corresponding frames A through P. In this example, it Is assumed that a first cut consists of frames A to B 
relating to a car, a second cut consists of frames C to N relating to a girl and a house, and a third cut 
consists of frames O to P relating to a building. The second cut Includes a plurality of shots. For example, a 
50 group of the frames C to F is referred to as a zoom-In shot, a group of the frames G to I is refenred to as a 
panning shot, a group of the frames J to N is refen-ed to as a zoom-out shot, respectively, in accordance 
with corre^onding camera operations. 

The digest display control section 308 selects at least one Image Included In the moving image based 
on the frame detected by the shot bre^polnt detecting section 307, and determines how the selected 
55 images, display as a digest. The selected images, sure conveyed lo the output section 309, where the 
selected images are displayed as a digest. 

Hereinafter, the operation of the motion vector detecting section 304 will be described with reference to 
Figures 19A and 19B. In Figure 19A, M lines and N lines are an-anged in a horizontal direcfion and a 

19 
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vertical direction, respectively through the entire screen. The pent of '"t^'^^^" IfJn the no nt of 
iines and the respective N lines shows me position of a motion vector to be detected Herem the pom of 
S inLsection is referred to as a lattice point The point of inlersectfon of the l.ne of the M hnes and the 
itft line of the N Knes is represented as a lattice point (i, j), wherein 1 S i £ M. 1 £ i s N 
^ "n the present example, a motion vector at a lattice point is determined "^V -'^f "9 °' 
representative points around the respective lattice points, and by repr^entafve P°'"t, 
7oZ a zoomed-up portion including a lattice point (I. j) of Rgure 19A to clarify a relationship between the 
DofntT i) and (2-m - l)-(2.n - 1) representative points located around the lattice pomt. Herem, a 
rf resen^lt ^olnt kposad on\he Rth line in the horizontal direction and the 1th line in the verjc^ 
direction is referred to as a representative point (i, j. k. 1). wherein -m S k S m. -n S 1 ^ n. As is seen from 
Figure 19B. the representative point ff, j. O. O) is equal to the lattice point (i. j). ^. ^ . . ,, 

Figure 20 shows a configuration of the motion vector detecting section 304, which is he pful for 
clarifyrng how a motion vector is obtained. A video signal is input to the motion vector detecting section 304 
from the moving image recording medium 303. It is assumed that the video signal will '"put onoe for 
; every r frames, there'r is a prescribed number. Herein, an image at the time o t is -Jerred to an image 
of the 0th frame, and an image at a time of (t + r ) is referred to as an image of the (30-)tMrame . 

Nov/ it is assumed that an image of the Rth frame is input to the motion vector detecting sec ion 304. 
The in jt Uage is first passed through a band pass filter included in a BPF 331 Herein, the value of an 
image at the point of coordinates (x. y) after the BPF processing is represented by l(x, y). 
, A representative point value memory section 332 stores the valued the representative po-m 
image r frames before, i.e. an image of the (R - r)th frame, after the BPF processing. Specrfically. the 
following value at the representative point (i, j. k, I) is stored: 

Y(i. j. k, I) = !(pos x(i, k). pos y(j. D) 

where. 1 s i S M. 1 £ j ^ N. -m ^ k S m, -n S I S n 
pos x{i. k) represents an x coordinate of the representative point (i. |. k. I) 
pos y(i, 1) represents a y coordinate of the representative point (i. U. I). . 
Tmatching section 333 receives the image l(x, y) after the BPF processing from the BPF 331 and 
10 receives the representative point value Y(l. j. k, 1) from m representative pomt va ue recording section 332 
and thus determines a motion vector at respective lattice points by the use of the ^^^P/^^f^^^'ve point 
. matching technique. Specifically, a motion vector (g. h) is determined by searching g and h which make the 
f«a value minimum with Aspect io the lattice point (i, j) the g and h being within the range of (2.G).- 
(2. H), Wherein. -G S g S G, -H £ h SH: 



j, k, 1) - l(pos x(i, 0) 

pos y(j, 0) 



0) +") 



The contents of the representative point value memory section 332 is updated ^^^^ P^™9 °J 
the matching section 333 is completed. More specifically, tiie representative pomt value of the Rth image 
afTer t^e BPF processing is recorded by using the following coordinates of the representative point stored in 
the representative point position memory section 334: 

pos x(i. j, k, I), pos y(i, j, k, I)) 

1 ^ i < M 1 S i £ N, -m 5 k S m, -n ^ I £ n, . j „ 

as" is described above, the motion vector can be determined ti-om two images, an input image and an 
image r frames before the input image. ^, u h=ir^*,.i fr>,- 

Next, the operation of the camera operation assuming section 305 will be described, which is helpful for 
clarifying how the camera operation is assumed from a motion vector. 

Some camera operations can be assumed from the moving images. For example, a change of a 
direction of the camera in the horizontal or vertical direction, such as a panning °P^^^^°" 
operation, a change of an angle oi view of the camera, such as a zooming operaton, a change of a pos, on 
of the camera in the horizontal, vertical, fore or back direcUon, such as a tracking operation, a booming 
operation, or a dollying operation, can be assumed from the moving images. For convenience, a ^^^J^^ 
^suming three kinds of camera operations, the panning operation, tilting operation, and zooming operation 
will be described below. 
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First, it will be described how a point wliich is projected to an imaging plane of the camera is moved in 
accordance with the above three kinds of camera operations. 

Figure 21 shows a postional relationship between the imaging pfane of the camera and an object. The 
. coordinates in a three-dimensional space of the camera are represented by (x, y, z), and the coordinates of 
s the two-dimensional imaging plane are represented by (X, Y). It is assumed that Ihe camera is disposed at 
the origin of the three dimensional coordinates, and the optical axis of the camera is z axis. It is also 
assumed that the imaging plane is located at a position where z = F (F: a focal length). An arbitrary point 
having the coordinates u1 = (x1, y1, zl) is projected onto a point having the coordinates U1 = (XI, Y1) on 
the imaging plane. The relationship between the coordinates of the object and the coordinates of the 
10 imaging plane is represented by the following equation (6): 

X1 = F-x1 / zl 

Y1 = F.y1 / zl (6) 

is With reference to coordinates of Rgure 21, it will be described how the coordinates of the object 
projected onto the imaging plane are moved in accordance with the zooming operation. 

Figure 22A illustrates the cfiange of the coordinates of the ol^ect projected onto the imaging plane in 
accordance with the zooming operation. As is shown . in Rgure 22A, when the focal length is changed from 
F to F', the projection of the object at the coordinates of u1 is changed from U1 = (X1, Y1) to U2 = Q<2, 

20 Y2). According to the equation (8), U2 satisfies fSie following conditions: 

U2 = UI.F'/F = j.UI 
where, f = F / F. 

25 Next, with reference to Rgure 22B. rt will be described how the coordinates of the object projected onto 
the imaging plane is moved in accordance with the panning operation or the tilting operation. The panning 
operation is defined as an operation for rotating the camera with respect to the y axis, while the tilting 
operation is defined as an operation for rotating the camera with respect to the x axis. As is shown in Figure 
22B, in a case where the camera is rotated by 9x with respect to the x axis, the coordinates u1 of the 

30 object in Uie space is moved to the coordinates u3. Herein, the coordinates u3 satisfies the following 
equation (7): 



When assuming that the rotation angle ex is sufficiently small, the coordinates U3 = (X3, Y3) on the 
. imaging plane after the movement are represented as follows from the above equation (6): 

X3 = XI, Y3 = Y1 + F.flx 

When the camera is rotatable both on the x axis and the y aw's, a relationship between the coordinates 
Ul before the operation and the coordinates U3 after the operation is more generalized so as to derive an ' 
equation as fellows: 

so U3 = Ul + P 



wherein, P = (px, py), and 

px and py are rotation components on the x axis and the y axis, respectively. 

With respect to the general camera operation by a combination of the zooming, panning, and tilting 
55 operation, the coordinates U1 = P<1, Y1) before the operation and the coordinates U' = (X", Y") after the 
operation satisfy Hie foiiowing equation: 

U' = f-U + P 
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Hereinafter, f is referred to as a zooming factor, and P is referred to as a rotation vector. 

Accordingly, it is possible to assume tHe operation amount of the camera by determining the zooming 

^^^^He^reSl^^Ter^'S^dScribed how to determine the zooming factor and the rotation vector based on a 
motion vector detected by the motion vector detecting section 304. It .s assumed that U,j denotes 
coordinates of the lattice point (i, j) in the two-dimensional imaging plane, and v, denotes a motion vector 
detected by the motion vector detecting section 304. ^ n >v. !oh-^= r,r^lnt n i^ 

If the camera operation caused the zooming factor f and the rotation vector P, the lattice point (i, j) 
would be moved to a position represented as follows: 

U'i,j{f, P) - f-Uij + P. 

Accordingly, in order to assume the actual f and P based on the actual camera operation, II is required 
to caTcuS an ;ror between the coordinates U',(f, P) and the actual position U-', and to determine f and 
P so that the error is made minimum. THe error is represented by the following equation: 

E (f, P) = £ (U'i,j(f, P) ■ W'^i)" 

where, tlie actual position is represented by U"*'ij = Ujj + .vij. , , ^ c 

The error E Is represented by a quadratic equation with respect to f and P. Accordingly, f and P are 
determined so that the error E is made minimum as follows: 



Herein {• •> denotes an inner product. , 
The camera operation assuming section 305 receives the motion vector v, and the lattice point position 
U. from the motion vector detecting section 304. and calculates the zooming factor f and the rotation vector 
P n accordance with the equation (8), thereby finally assuming respective operation amount of *8 zooming 
■ panning, and tilting operation. The determined zooming factor f and rotation vector P are recorded m the 

camera operation receding medium 306. ^ 1..^ «f 

Next.The operation of the shot breakpoint detecting section 307 will be described. When a mode of 
displaying a digest is set. the shot breakpoint detecting section 307 reads the zooming factor f and the 
Si^n victor P from the camera operation recording medium 306. arKi then detects a shot breaKpoin 
based on the zooming factor f and the rotation vector P. Herein, the shot Is defined as a group of .mages 
corresponding to the same continuous camera operation, such as a close-up shot by zooming m an object 
a lona^ot for a Tar-off scene or the like. Accordingly, the shot breakpoint can be detected if a change point 
of the camera operation, such as a starting point of the zooming operation, and an end point of the panning 
operation is found. However, ther^ are cases where the camera is moved slightiy and the like due to a na^ 
int Sal camera operation. Therefore, herein, only drastic changes are f f ^^S^^^^^^ 
camera operation is considered to be a point where the change amount of f and/or P exceeds a certam 

'''TreinafSlr: it will be described how to detect a shot breakpoint based on the zooming factor f and the 
rotation vector P. Rgure 23A shows an exemplary configuration of the shot breakpoint detecting section 

Herein it is assumed that camera operation amounts f(R), px(R). and py(R) with respect to the image of 
the Rth frame are stored in an operation amount memory section 361, and the camera operations are 
, recorded at every r frames into Ihe camera operation recording medium 306. In tf)is case, .he shot 
• breakpoint detecting section 307 reads operation amounts f(R - r). px(R . r), and py(R - r) of the image 
of the (R + r)th frame from the camera operation reconlmg medium 306. 
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A variation detecting section 362 determines absolute values of the differences |f(R) - f(R + r)|, ipx(R) - 
px(R + r)|, and |py(R) - py(R + r)| of the between the operation amount of the currently input image and 
the operation amount of the image r frames before, mVti respect to the zooming factor f, x-cwnponent of the 
rotation vector px, md y-component of ^e rotaiion vector py. respectrvely. A judging section 363 compares 

s three output values from the variation delecting section 362 with the respective threshold values. When at 
least one of the absolute values of the differences of the operation amounts is not less than the threshold 
value, the judging section 363 judges that a change of the camera operation occurs at the (R + r)th frame 
of image, and detects the (R + r)th frame as a shot bre^point. After the operation amount of the Rth frame 
is used by the variation detecting section 362, the contents of the operation amount memory section 361 is 

10 updated so as to store f(R + r), px(R + r), aiKl py{R + r) are stored. A threshold value setting section 364, 
where the threshold value to be used in the judging section 363 is set by the user's input operation, is used 
for changing the threshold value. 

Thus, tiie shot breal«pdnt. detecting section 307 is operated for an Image of a frame. Next, an 
exemplary procedure for detecting the shot breakpoint wilt be described. Rgures 24A through 24E show 

IS how the actual operation amount is changed as time elapses. Figures 24A through 24C show a relationship 
between the frame number and an operation amount and the frame number and a variation of the operation 
amount, with respect to the zooming factor f, x-component of the rotation vector px and the y-component of 
the rotation vector py, respectively. In Figures 24A through 24C, two horizontal broken lines shows 
threshold values. The frame numbers where the variation amount exceeds the threshold value are indicated 

20 with an arrow. Thus, as is shown in Figure 24D, five shot breakpoints (1 ) through (5) are detected. 

As is described above, a shot breakpoint can be detected by comparing the respective variaticm of the 
operation amount wth the respective threshold values. 

In addition, a shot breakpoint may be detected by comparing the respective operation amount with the 
respective threshold values. For example, if an operation amount of the zooming factor f increases arKl th@n 

25 exceeds a predetemnined threshold value at a frame, then the position of the frame represents a shot 
breakpoint (See, (6) of Figure 24D). In order to compare the respective operation amount with the 
respective threshold values, the shot breakpoint detecting section 307 may be modified so as to add an 
operation amount detecting section 362' as shown in Rgure 23B. In this case, when at least one of the 
absolute values of the differences of the operation amounts and the operation amounts is not less than the 

30 threshold value, the judging section 363 judges that a change of the camera operation occurs at the (R + 
.r)th frame of image, and detects the (R + r)lh frame as a shot breakpoint. 

Hereinafter, the operation of the digest display control section 308 wilt be described. Herein, the digest 
display method is controlled on the basis of the shot breakpoints detected by the shot breakpoint detecting 
section 307. In the present example, the digest display is performed by selecting the images of a 

35 prescribed number of frames at every shot breakpoint. 

Figure 25 shows an exemplary configuration of the digest display control section 308. In Rgure 25, a 
counter 382 is a memory for storing a single natural number, wherein 0 Is set as an Initial value. It is 
assumed that a current camera operation processed by the shot breakpoint detecting section 307 
corresponds to an image of the Rth frame. 

40 The shot breakpoint detecting sectioi 307 judges whether the cunrent Rth frame is a frame con-espond- 
ing to a shot bre^polnt or not. The judgment result is input to the counter control section 381, When the 
counter control section 381 receives the judgment result indicating that ttie current Rth frame is a frame 
corresponding to a shot breakpoint from the shot breakpoint detecting section 307, the contents of the 
counter 382 is changed to a prescribed value of TH. On the other hand, when the counter control sectfon 

45 381 receives the judgment result indicating that the current Rth frame is not a frame corresponding to a 
shot breakpoint from the shot breakpoint detecting section 307, the contents of the counter 382 is 
decrements by one. However, if the contents of the counter 3S2 is 0, the contents remain unchanged. 

After the "processing of the counter control section 381, it ts determined whether the image is displayed 
in the image selecting section 383. Specifically, when the contents of the counter 382 is larger than 0, the 

60 image of the Rth frame is selected as a display image, and then transmits the video sign^ to the output 
section 309. The output section 309 displays the transmitted image as a digest. Thereafter, a synchronous 
section 384 sends information that the processing for the image of the Rth frame is completed for the 
moving image recording medium 303 and the camera operation recording medium 306, and then initiates 
the processing for an image of the next frame. 

65 When the above processing is performed in the example shown in Figures 24A through 24D, the result 
shown ill Figure 24E can be obtained, wherein an image of a frame hawng a value of 1 is displayed as a 
digest image. 
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As is described above, the camera operation during the recording of a moving Image Is assumed, and 
the images to be displayed are selected on the basis of the camera operations, and thus the digest display 
to meet the user's intent can be performed. 

In the present example, the motion vector is detected according to the representative matching 
technique, but it can also be detected by other techniques such as a block matching technique and a 
characterized point matching technique. 

In the present example, the zooming operation, the panning operatton and the tilting operation are 
assumed, but any other camera operation such as a dreaming operation, or the like may also be assumed 
to obtain the same effects. 

In the present example, the digest display method in which the images are respectively displayed for 
the prescribed time at every shot breakpoint is adopted, but it can employ any other methods to obtan the 
same effects, such as a method in which the screen is multi-divided, and a plurality of images conresppnd- 
Ing to respectfve shot breakpoints are simultaneously displayed on the screen. 



A recording medium for recording a moving image according to a seventh example will be descnbed 
with reference to Figure 26. Figure 26 exemplarily shows the recording medium in the form of a video tape, 
but the moving image record medium of the present example may be in other form of a video disk or anJC 
nrjemory or the like. , ^ ^- c 

As is shown in Figure 26, The video tape acconding to the present invention includes a portion for 
recording a video signal per frarr«e unit and a portion for recording extracting information on the 
representative image corresponding to the video signal. Herein, the extracting information of the representa- 
five image includes camera operation information, image processing information, and sensing information. 
The video tape includes a portion for recording at least one of the camera operattons information, the image 
processing iniormation, and ttie senang information. ^. ^ 

■ : .>The camera operation information means in formation obtained by detecting a camera operation made 
by the user during the recording a moving image, such as information on a recording start operabon and a 
zooming operation. The information on the recording start operation means information indicating a point of 
the .recording start by martting the point with a flag. TTie information on the zooming operation means 
information indicating the rate of the zooming. Both the information on the recording start operation and the 
Information on the zooming operation can be detected on the. basis of the button operations of the video 

^^^The image processing information means information obtained by automatically or manuaily extracting 
and/or processing Ihe video signal. The image processing infbnmation includes, for example, a frequency 
and the size of the high frequency component of a video signal detemiined to perform the focus control; a 
difference value between the frames In luminance signals and color signals; a position and a size of an 
object region extracted from the video signal; a backlight state and too excessive light state; y correction 
value; and a color temperature. ^ ■ 

The sensing information means information obtained by detecting the state of the camera during the 
recording of fhe moving image by the use of any sensor. The sensing information include, for example, 
camera movement due to the panning operation detected by an angular speed sensor, or light intensity 
from an object detected by a light intensity sensor, an opening degree of the. diaphragm of the lens 
detected by an aperture opening sensor, and a focus distance detected by a lens location detector. 

According to the recording medium provided with sudi representative image extracting information, it is 
possible to extract the representative images among a plurality of images included in a moving .mage as is 
described lat^. 



Figure 27 shows a configuration of an apparatus tor generating a digest of a moving image according to 
an eighth example of the FMresent invention. , 

In the present example, a video signal and extracting information for extracting a representative image 
are recorded into a record medium. Then, the apparatus reads the extracting information from the record 
medium, and then evaluates the extracting information. On the basis of the evaluation, the apparatus 
extracts a representative image, as a static image, among images included in a cut. Herein, a cut rneans a 
group of successive static images from a frame at which the recording is started Id a frame af which the 
recording is ended. 
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As is shown in Figure 27, tiie apparatus includes an input section 401 for input a reproduced 
Infonmation, a camera operation information reading secSon 402. an image processing information reading 
section 403, a sensing information reading section 404, a video signai reading section 405, a representative 
image extracting section 406, a refMresentative image managing section 407, a represenlalive image 
6 memory section 408, and an output section 409. However, il is not necessary for tlie apparatus to inciude 
all of the camere operation Infonmation reading section 402, ttie image proccessing informafion reading 
section 403 and the sensing information reading section 404. The apparatus may inciude at least one of the 
camera operation information reading section 402 and tlie image processing informaUon reading section 
403. 

10 Hereinafter, the operation of the above respective sections will be described In detail. 

The information recorded in a recording medium is reproduced and input Into the input section 401. 
The camera operation information reading section 402, the image processing information reading section 
403, and an sensing information reading section 404 read out camera operation infonrtalion, image 
processing information, and sensing infonmation, respectively, from the reproduced signal input into the 

15 input section 401. In the present exsuntjple, each coded information is decoded. 

The representative image extracting section 406 detects a t"me period from a recording start to a 
recording, end based on at least one of the camera operation information and the image processing 
information. For example, the recording start operation information included in the camera operation 
information is used to detect the point of the recording start. Next, on the basis of at least one of the 

20 camera op^ation information excluding the recording start operation information; the image processing 
information; and the sensing information; the representative image extracting section 406 evaluates whether 
an Image included in each cut is suitable as a representative Image for the cut An image evaluated to be 
suitable for the representative image for the cut is input to the representative image managing section 407 
from the video signal reading section 406, and is stored in the representative image memory section 408. 

35 The represafitative image stored in ttie representative image memory section 408 Is read out and displayed 
by the output section 409 such as a display and a printer. Ttie operation of the representative image 
extracting section 406 will be described later in detail. 

in the present example, the information for extracting the representative image as well as the video 
signal are previously recorded in the recording medium, and the extracting information for the representa- 

30' tive image is read out and then the representative image is extracted from the recording medium. However, 
even If all or a pari of the information for extracting the representative images is absent in the recording 
medium, the video signal recorded in the recording medium can be processed so as to acquire the 
information for extracting the representative image, and then ttie r^resentative image can be extracted on 
the tjasis of the acquired information. Such a principle is late described in detail with respect to an 

35 apparatus for generating a digest of a moving image according to a ninth example. 

Example 9 

Figure 28 shows a part of a configuration of an afH^'^'atus for generating a digest of a moving image 
40 according to the ninth example of the present invention. This part is a circuit for acquiring information for 
extracting a representative image from a video signal instead of reading the information from a recording 
medium. 

As is shown In Rgure 28, the circuit includes a cut-change detecting section 413, a camwa work 
detecting section 414, an object information detecting section 417 and a focus information detecting section 

45 420, The frame difference value detecting section 410 includes a membry 411 and a change amount 
detecting section 412. The camera work detecting section 414 includes a motion vector detecting section 
415 and a camera work parameter assuming section 416, The object iniormation detecting section 417 
includes a moving area detecting section 418 and an inner area feature amount extracting section 419. The 
focus information detecting section 420 includes a high pass filter 421 and an average calculating section 

so 422. 

The operation of tiie above respective sections v/ili be described in detail. 

Rrst, the operations of the frame difference value detecting section 410 and the cut-change detecting 
section 413 wlli be described. The frame difference value detecting section 410 includes a memory 411 for 
delaying a moving image signal by one frame; and a change amount detecting section 412 for determining 
6s a difference v^ue of .moving image signals between the respective successive frames. The frame difference 
value detecting section 410 calculates the difference in a signal between tiie frames for each of 
con^sponding pixels and sum up tiie diff^^ce over all corresponding pixels, and outputs tlie summed up 
difference as a frame difference value. To determine the difference value, a luminance value, rgb value can 

25 
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The cut-change detecting section 413 compares the frame difference value with a threshold value, and 
judges that a cut-change occurs when the frame difference value exceeds the threshold value. This is 
because the frame difference value is higher as the contents of images between successive frames is 
5 changed more drastically. A video camera causes a cut-change when the recording start operation is made. 
■ so that the recording start operation can be assumed by detecting the cut-change from the image signal. 
The cut-change detecting section 413 outputs a recording start operation information at a time when it is 
detected that the frame difference value is larger than the threshold value. The stmcfure of the frame 
difference value detecting section 410 mentioned above is only one example. For example, 8ie frame 
JO difference value detecting section 410 may have the structure shown in Figure 29. 

In Figure 29, the frame difference value detecting section 410 includes a color histogram detecting 
section 444 for determining a color histogram per frame of the moving image signal, a histogram memory 
446 for storing the determined histogram, and a histogram difference detecting section 446 for detectmg a 
difference of color histograms between the respective two successive frames. According to the structure 
15 shown in Figure 29. the frame difference value is not compared with the threshold value per pixel but it is 
compared with the threshold value per frame, in addition, when a screen is divided into a plurality of blocks, 
the frame difference value can be compared with the threshold value per bloci<. 

The operation of the camera wori< detecting section 414 is the same as that of the camera operation 
assuming section 305 of the apparatus for generating a digest of a moving image according to the sixth 
20 example, so that the description thereof is omitted. ^ u- 

Next, the operation of the object information detecting sectiors 417 will be described. The object 
information detecting section 417 extracts object Information such as a portion, a size and a color of the 
object while the video camera is tracking the object. When the camera work detecting section 414 detects a 
panning operation, and the moving area detecting section 418 detects a moving area, then the inner area 
£5 feature, amount extracting section 419 extracts a feature amount in the detected moving area. 
The operation of the moving area detecting section 418 will be described below in detail. 
Motion vectors vsj at respective lattice points in the screen detected by a motion vector detecting 
section 415. and a motion vector Vp due to the panning operation detected by the camera work detecting 
section 414 are input to the moving area detecting section 418. The . moving, area detecting section 418 
30 extracts, lattice pants satisfying the following equation (9). and extracts an area different from tirat of the 
motion vector Vp due to the panning operation based on a connecting relationship between tiie extracted 
lattice,,F?pints: 

Ki-Vp|>. (9) 

35 

Wherein, « is a prescribed value. .u- 
The inner area feature amount extracting section 419 extracts a center, a dimension and color of the 
area as an inner feature amount from the area detected by the moving area detecting section 4m 

Next the operation of the focus information detecting section 4a0 will be described. The focus 

40 information detecting section 420 detects an out-of-focus state of the image base on an amount of the high 
frequency compraients of tiie image. The value of the high frequency components of the video signal 
becomes smaller in the out-of-focus state. The high pass filter 421 picks up the high frequency components 
of the image, and the average caiculatirrg section 422 determines an average value of the high frequency 
components included in the entire screen or in a prescribed region of the screen. . 

45 Thus it is possible to acquire information for extracting a representative image by processing the video 
signal as' is described above. Even though a y correctioh value, color temperature, backlight and excessive 
tight state, and the light intensity from an object are not mentioned in the present example, but such 
information can also be acquired by processing the video signal in tiie same manner. After the information 
for extracting the representative image is acquired, the representative image Is extracted on the basis of the 

50 thus acquired infonnafion in a manner sirhilar to that of the eighth example. 

According to the apparatus mentioned above, the representative image extracting information is 
acquired on the basts of tiie video signal read out from the recording medium when the information for 
extracting the representative image is absent in the recording medium. However, it is also possible to 
acquire the representative image extracting information on the basis of the video signal received from an 

65 imaging element during the recording of a .moving image. In ESiis case, the device has the same structure as 
that of Figure 28. Needless to say, but it is not necess^y to acquire the representative image extracting 
information from the video signal in the case where ttiere is the representative image extracting information 
capable of being detected by a sensor provided in the video camera. Furthermore, the representative image 
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extracting information acquired during the recording of a moving image may be recorded into the recording 
medium together wrttln tlie video signai. 

Exampte 10 

Figure 30 shows a configuration of an apparatus 427 for recording a moving image in a recorcling 
medium according to a tenth exampte of the present invention. In the present example, the apparatus 427 is 
incorporated in a video camera. The apparatus 427 determines a static image to be extracted as a 
representative Image during the recording of a moving image, and records information for the static image 
and a video signal into a recording medium. 

As is shown in Figure 30. the apparatus 427 includes a camera operation information acquiring section 
428, an image processing information acqiMring section 429. a sensing information acquiring section 430, a 
video signal acquiring section 431, a representative image extracting section 432. a static image recording 
section 433, and a video signal recording section 434. However, it is not necessary for the apparatus to 
include all of the camera operation information acquiring section 428, the image processing ihfomiation 
acquiring section 429 and the sensing information acquiring section 430. The apparatus may include at 
least one of the camera operation information acquiring section 428 and the image processing information 
acquiring section 429. 

The operation of the above respective sections will be described below. 

The camera operation information acquiring secfion 428 acquires information on the recording start 
operation and zooming operation during the recording of a moving image. The Information on the recording 
start operation means information indicating the point of the recording start by marking the point vi/ith a flag. 
The information on Bie zoomirig operation means information indicating a rate of the zooming. Both the 
information on the recording start operation aid. the zooming operation can be detected on ttie basis of the 
button cperations of the video camera 

The image processing information acquiring section 429 acquires information obtained by processing 
the video signal Imaged by an imaging element. The image processing information includes, for example, 
the frequency and the size of the high frequency component of a video signal determined to perform the 
focus control; the difference value between the frames in luminance signals and color signals; the position 
artd the size of an .object region extracted ircm the video signal; the backlight state and too excessive light 
state; y cortecfion value; and the color temperature. 

The sensing information acquiring section 430 acquires information obtained by detecting the state of ■ 
the camera during the recording of the moving image by the use of any sensor. The sensing information 
includes, for example, camera movement due to the panning operation detected by an angular speed 
sensor, or light intensity from an object detected by a light intensity sensor, an opening degree of tfie 
diaphragm of the lens detected by an aperture opening sensor, and a focus distance detected by a lens 
location detector. 

The representative Image extracting section 432 detects the time period from the recording start 1o the 
recording end based on at least one of Vne camera operation information and the image processing 
infbnmalion. .For example, the recording start operation information Included In the camera operation 
information is used to detect the point of the recording start. Next, on the basis of at least one of the 
camera operation information excluding the recording start operation infomiation; the image processing 
information; and the sensing information; the representative image extracting section 432 evaluates whether 
an image included in each cut is suitable.as a representative image for the cut. InfonmatlOT on an image 
evaluated to be suitable for the representative image for the cut is recorded in a static image recording 
medium in the recording medium 435 by the static mage recording section 43a The operation of the 
representative image extracting section 432 will bs described later in detail. 

Hereinafter, the static image information v/hich is recorded in the recording medium 435 by the static 
image recording section 433 will be further described. The static Image information Includes a static image 
itself which is judged to be appnapriate as the representative image by the representative Image extracting 
section 432, an Image processed by an Image process such as a reduction, information indicating a position 
at which the static image is recorded on the recording medium, or a flag added to a moving Image to mark 
a position of the static image in the moving image. 

in the case where the static image Information is the static Image itself, or a reduced image, the static 
image or the reduced image is reconjed at a position different from a position at which the moving image is 
recorded by ttie video signal acquiring section 431 and the video signal recording section 434. AlternaUvely, 
the static image or the reduced Image is recorded into a recording medium other than a record medium in 
which the moving image Is recorded. For example, in the case where me recording medium is a video tape, 
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only the static image information (i.e. the static image or the reduced image) is recorded into a top portion 
or an end portion of the video tape. Altematively. only the static image information is recorded into an IC 
mennory provided separately frcNnn the video tape. ... , u. ^t^r.r- 

in the case where the static image information is information mdicatmg a position al which the static 
image is recorded on the recording medium, the static image information is recorded at a position different 
from the position at which the moving image is recorded by the video signal acquiring section 431 and the 
video signal recording section 434. Alternatively, the static image or the reduced image is recorded into a 
recording medium other than a record medium in which the moving image is recorded. 

In the case where the static image information is a flag added to a moving image to mark a position of 
the static image in the moving image, the static image information is recorded at the same position as the 
position at which the moving image is recorded by the video signal acquiring section 431 and the video 
signal recording section 434. For example, the static image information is recorded at the top portion of the 
video signal to be recorded per frame unit. ^ ^ ■ ,u„ ,„^„.Hin^ 

AS is described above, the video signal recorded by the video camera is recorded into the recording 
; medium, and a representative image is extracted among the recorded moving images^ Then, finally the 
static image information of the representative image is recorded into the record medium. Thus, it is possible 
to read out the recorded static image information and output the representative image to a display or a 

'^"^ Hereinafter it will be described how to extract a static image from the moving images and how to 
J evaluate a static image in the representative image extracting section 406 shown in Figure 27 and the 
representative image extracting section 432 s*own in Hgure 30. . 

The representative image extracting sections 406 and 432 automatically extracts a representative image 
as a static image from a plurality of images included in the moving image which are successively recorded 
during the period from the recording, start to the recording end. Herein, the representative image is defined 
5 as an image selected based on the evaluation relative to the user's intent, the state of the recorded moving 
image and the state of the object. . „ » 

:-.- The user-s intent generally is reflected by the camera operation such as a zooming operation and a 
panning operation. Specifically, when a zooming-in operation is being made, the target object is assumed to 
be present in the screen, and therefore, the image at that time may be considered to be important, When a 
0 panning operation is being made, the scene to be recorded is assumed to be being changed to another 
one and the image at that time may be considered to be unimportant. However, if an tracking obiect is 
present during the panning operation, the image may be considered lo be important. Thus, the user s intent 
is .preferably assumed from camera work so that important images can be extracted as representative 

55 ""^tS slate of the recorded moving image means a, state such as an out-of-focus state du® to a 
misopeialion of the focus control during the recording of a moving image, an excessive light or backlight 
state due lo a misoperaiion of the iris control, a state where y correction cannot be performed properly; or a 
state where the aperture and/or the focus is adjusted. Such a state of the image can be judged on the basis 
of the informaUon on the focus control and/or the iris control during the recording of a moving image or 

« infonnation on a 7 correction value. Even if such information is not available, the state of the image can be 
■ obtained by processing the video signal. Thus, by evaluating the state of the respective images, images 
having a good state are preferably extracted as the representafive images. 

The state of an object means the location and/or size of the object, the state of an obstacle when the 
obstacle {a person or the like) passes through the scene, a state where a flashlight or the like is snapped or 
45 a state where the object is spotlighted, in view of such a state, for example, an image where the object is 
located at the center, the size of the object is large, and/or no obstacle is present is preferably selected as 
the representative image. An image where the flashlight is snapped is preterably not selected as a 
representative image. An image where the object is spotlighted is preferably selected as the representative 

so '"'The position and the dimension of the object are detected in the same manner as described in the ninth 
■ example with respect to the object information detecting section 417 of Figure 28. The flashlight and the 
obstacle are detected on the basis of a frame difference value determined by the frame difference value 
■ detectinq section 410 of Figure 28. When the flashlight is snappec, the frame difference value is drastically 
changed so that it is possible to detect whether the flashlight :s snapped or not on the basis of a 

56 prescribed threshold value. On the other hand, when an obstacle is present, the frame difference value ,s 
changed from a frame where the obstacle enters to a frame where the obstacle goes out. Therefore. «ie 
obstacle is considered to be in the screen when the frame difference value exceeds the threshold value 

. once and then exceeds it again in a prescribed period of time. In addition, it is possible to detect whether 
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the objfect is spotlighted or not on the basis of the light intensity from the object. 

Hereinafter, a method for extracting a representative image according to the above-mentioned princi- 
ples. 

Figure 31 shows a configuration of the r^resentative image extracting section 432. As is shown In 

s Figure 31, the representative Image extracUng section 432 includes a weight adding section 436, a gate 
signal generating section 437, a gate section 438. and a maximum value detecting section 439. A zooming 
ratio and the object information are input to the weight adding section 436. where the respective signals are 
weighed and added. Herein, the object information means information obtained by tracldng the object by 
the camera, which is obtained on the basis of the position and the size of the object during the tracking 

10 operation. The object information has a larger value as the object is disposed nearer to the center of the 
camera and the size of the object becomes larger. The gate section 438 switches an on/off-state in 
response to a gate signal from fiie gate signal generating section 437. The maximum value detecting 
section 439 detects a maximum value among values which are input from the gate section 438. 

The gate signal generating section 437 generates a gate signal on the basis of a panning signal, high 

T5 frequency component value of a video signal, and a frame difference value or the like. 

It v*Fill be described hov/ to generate the gate signal with reference to Rgures 32A through 32E. Figure 
32A show a waveform of a panning signal, which has a value of 0 during the panning operation, and has a 
value of 1 during the non-panning operation. Figure 32B shows a waveform of a high frequency component 
value of the video signal, and the value becomes smaller as the image becomes out-of-focus. Figure 32C 

20 shows a wavefdrrin of a binary signal obtained by digitizing the high frequency component value shown in 
Figure 32B in accordance with the threshold value, and it is given a value of 0 if the high frequency 
component value is not more than ttie threshold value. Figure 32D shows a waveform of the frame 
difference value. Figure 32E shows a waveform of the value obtained by adjusting the frame difference 
value shown In Figure 32D on the basis of the threshold value, and it is given a value of 0 if the frane 

25 difference value is not less than the threshold value. Moreover, if the frame difference value shown in Figure 
, , 32D exceeds the threshold value once, and then exceeds it again within a prescribed period of time, the 
corresponding value of Figure 32E maintains the value of 0 from the time when the frame difference value 
exceeds the threshold value to the time when it exceeds the threshold value again within the prescribed 
period of time. When the frame difference value shown in Figure 32D exceeds the threshold value at only 

30 one time within the prescribed period of time, it is assumed that the image abnormally occurred by a . 
' flashlight or the like, so-that the corresponcfing value of Figure 32E will have the value of 0 only at that time. 
However, when the frame difference value shown in Rgure 32D exceeds the threshold value at a plurality of 
times, as such a case vriiere the c^stacle passes through the camera, it is as^med that the obstacle is 
present in the screen, so that the value.of Figure 32E remains at 0 during that time when the obstacle Is 

35 considered to be presarit By exeoiting the AND operation for the binary three signals shown, in Figures 
32A, 32C and 32E, the gate signal is generated. 

Thus, according to the structure shown in Figure 31, by extracting an image which has a maximum 
evaluation value as a representative image, it is possible to extract images where the object is zoomed-in, 
and/or the object is center-located among images excluding images recorded during the panning operation, 

40 the out of focused images, and the images where the flash light and/or tne obstacle is present. The 
maximum value detecting section 439 may detect the maximum value from the entire cut, or from a 
plurality of portions included in the cut. 

In the present example, one evaluation value is determined by the weight adding section 436, the gate 
signal generating section 437, and the gale section 438 on the basis of a plurality of input signals. However, 

45 tive present example is not limited to the above structure, but the evaluation value can be determined 
according to fuzzy inference rules or neural network rules. 

In the present example, information as to the 7 correction value, the iicht intensity from the object, the 
backlight or excessive light state, aperture opening, and/or focus length is not described, but such 
information may also be utilized in the present example. The gate signal can be generated so that images 

50 which are recorded when the y correction value, the aperture opening, and/or the focus length fluctuate, and 
images having the backlight or excessive light state are not extracted as Hie representative images. In 
addition, it can be detected whether me object is spotlighted or not on the basis of the fight intensity from 
tile object, so that the evaluation value may be set higher when it is detected that the object is spotlighted. 
In the present example, the evaluation Is performed with respect to all the moving images during a 

55 period from the recording start to the recording aid, and ihert the representative images are extracted. 
However, the evaluation may be performed with respect to only the images after a prescribed period of time 
after the recording start, and then the images where prescribed conditions are satisfied may be extracted as 
the representative images. Such a structure will be described below. 
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Figure 33 shows another configuration of the representative image extracting section. As is shown in 
Figure 33, the representative image extracting sectiorr includes a timer 440. a gate section 441. and an 
evaluating section 442. The timer 440 determines an elapsed period of time from the recording start, and 
aenerates a gate signal so that the gate of the gate section 441 is opened when a prescribed penod of time 
elapses after the recording start. The evaluating section 442 evaluates whether the high frequency 
component value of the focus which passed through the gate section 441, and the frame difference value 
satisfy prescribed conditions or not. The prescribed conditions in the evaluating section 442 are that the 
high frequency component value of the focus is not less than a threshold value, and the frame difference 
value is not more than a threshold value. The evaluating section 442 stops the evaluation process at the 
time when the conditions are satisfied, and extract a static image satisfying the prescribed conditions as a 
representative image. ■ , j-ii 

m the present example, only the high frequency component value of the focus and the frame difference 
value are used in the.evaluation process, but the other signals such as panning and zooming signals may 

^^°Rgure 34 shows another conflguralion of the representative image extracting section. In Figures 33 and 
34 tike reference characters refer to like parts ttiroughoul. and the description thereof will be omitted. 

' In the present example, the evaluation is performed on the basis of the number of images obtained by 
multiplying the number of images which are recorded during the period from the recording start to the 
recording end by a prescribed rate, and then images satisfying the prescribed conditions are extracted as 
the representative images. According to the structure shown in Figure . 34. images after an intermediate 
frame are evaluated. An intermediate frame detecting section 443 detects an intermediate frame of the 
frames which are recorded during the period from the recording start tp the recording end. and generates a 
gate signal so that the gate of a gate section 441 is opened art the time when the intermediate frame is 
detected. The operations of the gate section 441 and the evaluating secUon 442 are the same as those 
shown: In Rgure 33. so that the description thereof is omitted. 

As- is described above, the recording medium of the present invention, includes the moving image 
managBmeot information capable of readily managing the contents of the moving image, so that the user 
can easily search a desired scene. , ^ ■ v ^ * 

In addition, according to the apparatus for recording the moving image of the present invention, a cut- 
change^image is automatically delected among the moving image recorded into the recording medium. 
Then recording addresses on the recording medium where the cut-change image is recorded, a time length 
of the^ respective cuts and a change degree of the moving Image in the respective cuts are, automatically 
recorded as a management infonnation for mana^ng the moving image, so that the management 
information can automatically be supplied even for a large amount of moving image data. 

Moreover the apparatus for generating the digest of the moving image of the present invention can 
automatically 'generate the digest image for a rapid traverse operation suitable for the contents of the 
recorded moving image, on the basis of the time length for the digest input by the user, and the 
management information recorded into the recording medium. 

Furthermore, according to the system for generating the digest of the moving image, and the method 
for generating the digest of the moving image of the present invention, the state of the respective images 
can be assumed on the basis of button input signals, and a priority of the respective images is determined 
in accordance with the assumed state of the respective images. Then, on the basis of the thus determined 
priority digest images are selected and a display speed is adjusted. Consequently, the digest to reflect the 
user's intent during the recording of the moving image can be displayed so that the contents of the moving 
image can readily be grasped. * . * 

Furthermore, by automatically determining the privity fram the button input signals, the contents of the 
moving image can readily be grasped without any manual operation for previously assigning the pnonty and 
without any manual operation for determining the priority in reproducing the moving image, so that one can 
grasp the contents of the moving image by a simple operation as well as a rapid traverse operation. 

in addition, according to the apparatus for generating the digest of the moving image of the present 
invention the camera operation during recording operations such as a zooming operation and a panning 
operation can be assumed from a video signal. Then, display images are selected on the basis of the thus 
assumed camera operation. Accordingly, the digest to reflect the user's intent during the recording of the 
moving image can be displayed so that the contents of the moving image can readily be grasped. 

Furthermore, by automaticaliy selecting the cisplay images on the basis of the video signal, the 
contents ol the moving image can readily be grasped without any manual operation for previously assigning 
the priority without any manual operation for determining the priority in reproducing the moving image, so 
that one can grasp the contents of the moving image by a simple operation as well as a rapid traverse 
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operation. 

Moreover, the recording medium of tfie present invention includes image extracting information, thereby 
readily extracting representative images from tiie moving image recorded . Into the recording medium on the 
basis of the Image extracting information. 
5 Furthermore, the apparatus for generating the digest of the moving image, and thie method for 

generating the digest of the moving image of the present invention can extract static images as the 
representative images from the moving image on the basis of the image extracting information. With the 
static images, the contents of the moving image can efficiently and briefly be grasped. 

Furthermore, according to the apparatus for recording the moving image of the present invention, a 
70 video signal can be recorded in the recording medium, and representative images are extracted from the 
moving image. Then, static image informatitm of the representative images is recorded into the recording 
medium, and consequently the recwded static image Infomnation is read out so as to be output by a 
display or a printer at high speed. 

Various other modifications will be apparent to and can be readily made by those skilled in the art 
IB without departing from the scope and spirit of this invention. Accordingly, it is not intended that the scope of 
the claims appended hereto be limited to the description as set forth herein, but rather that the claims be 
broadly construed. 

Claims 

20 

1. A recording medium comprising: 

a moving image recording portion for recording a moving image having a plurality of cuts; and 
a management information recording portion for recording management information for managing 
said moving Image recorded into said moving image recording portion; 
25 wherein said management information recording portion includes a first portjon for storing first 

infomnation indicating a position at which one of said plurality of cuts is changed to another one of said 
plurdity of cuts in said moving image, and a second portion for storing second information indicating a 
priority of each of said plurality of cuts in said mowng image. 

30 2. A recording medium according to claim 1 , wherein said first information is an address on said recording 
medium, and said second information includes a time length of each of said plurality of cuts, and a 
degree of the change of said moving image wittiin each of said plurality of cuts. 

a An apparatus for recording a moving image onto a recording medium, comprising: 
35 a management information generating means for receiving a moving image having a plurality of 

cuts, and for generating first information indicating the position at which one of said plurality of cuts is 
changed to another one of said plurality of cuts in said moving image, and s«x»nd information 
indicating the priority of each of said plurality of exits in said mowng image, based on said moving 
Image; 

40 a first recording means for receiving sad moving image, and for recording said moving image into 

a recording medium; and 

a second recording means for receiving said first infomnation and ssad second information 
^ generated by said management information generating means, and for recording said first information 
and said second information into said recording medium. 

45 ■ 

4. An apparatus according to claim 3, wherein said management information generating means includes: 

a cut-change detecting means for detecting the image existing at a position where one of said 
plurality of cuts is changed to another one of said plurality of cuts in said moving image; 

a first detecting means for detecting a time length of a cut including said detected image based on 
50 said detected image; 

a second detecting means for detecting a degree of the change of said moving image within a cut 
including said detected image based on said detected Image; and 

a third detecting means for detecting an address on said recording medium where said detected 
image is recorded. 

5. An apparatus for generating a digest of a moving image, 

an input means for inputting a desired time length of a digest of a moving image having a plurality 
of cuts; 
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a receiving means for receiving management information for managing said moving image; 

an extracting means for extracting at least one cut from said plurality of cuts in said moving image, 
based on said desired time lengtiJ and said management information; 

a determining means for determining the number of frames to be reproduced for each of said cuts 
extracted by said extracting means, based on said desired time length -and said management 

'"'"TselTc'tin^means for selecting an image of frame determined by said determining means for each 
of said cuts extracted by said extracting means. 

6. A system for generating a digest of a moving image, comprising: 

a generating means for generating a moving image having a plurality of frames, and for generating 
an operation signal in response to an input received by said generating means during generating said 

""TprioXassigning means for assigning a priority to each of said plurality of frames as a function 
of said operation signal; ^- . 

a selecting means for selecting at least one Image of a plurality of images corresponding to said 
plurality of frames, based on said priority; and 

an output means for outputbng said at least one image selected by said selecting means. 

7. A system according to claim 6. wherein said generating means includes a camera, and said input is 
supplied to said generating means by an operator of said camera. 

a . A system according to claim 6, further comprising: ^ 

means for recording said moving image having said plurality of frames and said pnority cor- 
; : responding to each of said plurality of frames into a recording medium: and 

means for reading said moving image having said plurality of frames and said pnonty correspond- 
„i(, to each of said plurality of frames from said recording medium. 

9. A system according to claim 6, further comprising: ^ ^. ■ , 

, .;;., <. means for recording said moving image having said plurality of frames and said operation signal 

,. into a recording medium; and ^ ■ ■ „„. « 

„ means for reading said moving image having said plurality- of frames and said operation signal from 
said recording medium. 

5 10. A system according to claim 6, wherein said priority assigning means includes: 

means for storing a state of an Image coffesponding to a frame previous to an input current frarrie; 
means for receiving an input operation signal, arvd determining a state of an image corresponding 
to said input current frame as a function of said input operation signal and said state of said image 
corresponding to said previous frame; and . , * . vi 

0 means for determining the priority of said input current frame as a function of said state of said 

image corresponding to said input current frame. 

11 A system according to claim 6, wherein said priority assigning means includes: 

a plurality of first priority determining means, each of said plurality of first priority determining 
B . means including means for storing the state of sn image corresponding to a frame previous to an input 
current frame, means for receiving an input operation signal and for determining the state of an image 
corresponding to said input current frame as a function of said operation signal and said stale of said 
imag6 corresponding to said previous frame, and means for detemnining a first priority of said input 
current frame as a function of said state of said image corresponding to said input current frame; 
50 a weight calculating means coupled to each of said plurality of first priority determining means, for 

calculating the weight for said first priority determined by each of said plurality of first priority 
determining means; and . . ^ „ 

a second priority determining means for determining a second priority of said input correal frame 
based on said respective first priorities which are weighted by said weight calculating means. 

12. A method for generaUng a digest of a moving image, comprising the steps of: 
. generaUng a moving image having a plurality of frames; 
generating an operation signal in response to an input during generating said moving image; 
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assigning a priority to eacti of said plurality of frames as a function of said operation signal; 
selectir»g at least one image of a plurcdtty of images correspwiding to said plurality of frames, 
based on said priority; and 

outputting said selected at least one image. 

13. A meHiod according to claim 12, wherein said priority assgning step includes the steps of: 

storing the state of an image coffesponding to a frame previous to an input current frame; 
receiving an input operation signal, ^d determining a state of an image corresponding to said 
input current frame as a function of said input operation signal and said state of said image 

10 corresponding to said previous frame; and 

determining a priority of said input cun-ent frame as a function of s^d state of said image 
corresponding to said input current frame. 

14. A method according to claim 12, wherein said priority assigning step includes the steps of: 

ts (a) determining a first priority corresponding to a cunrent frame as a function of each of a plurality of 

input operation signals; 

(b) calculating a weight for each of said first priorities obtained by step (a); and 

(c) determining a second priority conresponding to said current frame based on ssJd first priorities 
weighted by the step (b). 

30 

is. An apparatus for recording a moving image into a recording medium, comprising: 

a generating means for generating a moving image having a plurality of frames, and for generating 
m operation signal in response to an input received by said generating means during generating said 
moving image; 

25 a priority assigfiing means for assigning a priority to each of said plurality of franties as a function 

of said operation signal; 

a recording rineans for said moving image having said plurality of frames and said priority assigned 
to each of said plurality of frames into a recording medium. 

30 16. An apparatus for generating a digest of a moving image, comprising: 

a priority assigning means for receiving an operation signal generated during generating a moving 
[mage having a plurality of frames, and assigning a priority to each of said plurality of frames as a 
function of said operation sign^; 

a selecting means for receiving said moving image, and for selecting at least one image of a 
35 plurality of images con-esponding to said plur^ity of firames, based on said priority; and 

an output rtxe&ns for outputting said at least one image selected by said selecting means. 

17. An apparatus for generating a digest of a moving image, comprising: 

a tnotion vector detecting means for receiving a moving image ftaving a plurality of frames, and for 
■ 40 detecting a motion vector between a predetermined number of frames of said pluraJity of frames; 

a calculating means for calculating at least one of a value of a zooming factor, a value of a rotation 
vector, a change degree of said zooming factor between said predetermined number of frames, and a 
change degree of said rotation vector between said predetermined number of frames, based on said 
motion vector; 

4S a shot breakpoint detecting means for detecfing a shot breakpoint of said mowng image, based on 

at least one of said value of said zooming factor, said v^ue of said rotation vector, said change degree 
of said zooming factor between said predetermined number of frames and said change degree of said 
rotation vector between said predetermined number of frames; 

a selecting means for selecting at least one image of a plurality of Images corresponding to said 
50 plurality of frames, based on said shot breakpoint; and 

an output means for outputting said at least one image selected by said selecting means. 

18. An apparatus according to claim 17, wherein said moving image having said plurality of frames is 
recorded in a recording medium, and said motion vector detecting means includes means for reading 

55 said moving image from said recording medium. 



19. An apparatus according to claim 17, wherein s^d calculating means includes means for recording at 
least one of said value of said zooming factor, said value of said rotation vector, said change degree of 
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said zooming lactor between said predetermined number of frames, and said change degree of said 
rotation vector between said predetermined numiaer of frames Into a recording medium, and said shot 
breakpoint detecting means includes means for reading at least one of said value of said zooming 
factor, said value of said rotation vector, said change degree of said zooming factor between said 
predeiermined number of franes. and said change degree of said rotaBcMi vector between said 
predetermined number of frames from said recording medium. 

20. A recording medium, comprising: 

' a moving image recording portion for recording a moving image; and 
a management information reconding portion for recording at least one of camera operation 
information obtained by detecting a camera operation made by an operator during recording said 
moving image by the use of a camera, image processing information obtained by processing said 
moving image, and sensing infomnation obtained by detecting a recording condition during receding 
said moving image by the use of a sensor provided in said camera. 

21. An apparatus for generating a digest oi a moving Image, comprising: 

an information receiving means for receiving at least one of a camera operation Information and an 
image processing information, said camera operation information being obtained by detecting a camera 
operation made by an operator during recording a moving image by the use of a camera, said image 
processing information being obtained by processing said moving Image; 

a stafic image extracting means for delecting the time period from the recording start to the 
recording end based on at least one of said camera operation infbmnation and said image procesang 
information, and extracting at least one static image of a plurality of static images included in said 
mowng image recorded during said time period from the recording start to the recording end based on 
at least one of said camera (^ration information and said image processing information; and 

an output means for outputting said at least one static image extracted by said extracting means. 

22. An apparatus according to claim 21. wherein said information receiving means further receives a 
" sensing information (Stained by delecting a recording condition during recording said moving image by 

the use of a sensor provided in said camera, and said static image extracting means extracts at least 
dne static image of said plurality of static images included in said moving image recorded during said 
. time,, period from the recording start to the recording end based on at least one of said camera 
operation information, said image processing information and said sensing information. 

23^ An apparatus for recording a moving Image onto a recording medium, comprising; 

an information acquiring means for acquiring at least one of a camera operation information and an 
Image processing infomnation, said camera operation information being obtained by detecting a camera 
operation made by an operator during recording a moving image by the use of a camera, said image 
processing information being obtained by processing said moving image; 

a specifying means for defecting the time period from the recording start to the recording aid 
based on at least one of said camera operation information and said image processing infonmation. for 
specifying at least one static image of a plurality of static images included in said moving image 
recorded during said time period from the recording start to the recording end based on at least one of 
said camera operation Information and said Image processing information, and for generating informa- 
tion on said specified at least one static image; and 

a recording means for recording said moving image and said information on said specified at least 
one static image into a recording medium. 

24. An apparatus according to claim 23, wberein said information acquiring means further acquires a 
sensing information obtained by detecting a recording cCHidition during recording said moving image by 
the use of a sensor provided In said camera, and said specifying means specifies at least one static 
image of said plurality of static images included in sad moving image recorded during said time period 

from the recording start to the recording end based on at iesst one of said camera operation 
information, said Image processing Information anc said sefising ir!forrn.3iicn. 

25. A method for generating a digest of a moving image, comprising the steps of; 

receiving at least one of a camera operation infomnation and an image procesang information, said 
c^era operation information being obtained by detecting a camera operation made by an operator 
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during recording a moving image by the use of a camera, said image processing information being 
obtained by processing said moving image; 

detecting the time period from ttie recording start to tlie recording end based on at least one of 
said camera operation information and said image processing iniormafion; 

extracting at least one static image of a plurality of static images included in said moving image 
recorded during said time period from the recording start to the recording end based on at least one of 
said camera operation information and said image processing information: and 

outputting said extracted at least one static image. 

26. A method according to claim 25, wherein said receiving step Includes receiving a sensing Information 
obtained by detecting a recording condition during recording said moving Image by the use of a sensor 
provided in said camera, and said extracting step includes extracting at least one static image of said 
plurality of static images included in sard moving image recorded during said time period from the 
recording start to the recording end based on at least one of said camera operation information, said 
Image processing information and said sensing information. 
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